JPH1173200A - Acoustic signal encoding method and record medium readable by computer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To encode a vocal analog acoustic signal by MIDI data. SOLUTION: The vocal analog acoustic signal is digitized by a PCM to be divided into unit sections U1-U7 and representative frequencies and representative intensities are calculated for every section. MIDI sign data C1-C7 having note numbers corresponding to the representative frequencies and having velocities corresponding to the representative intensities and having delta times corresponding to lengths of the unit sections are defined for every individual unit section. The sign data C1-C7 are separately arranged on two tracks and the sign data being on the both tracks are made so as to partially overlap each other on a time base by prolonging delta times of respective sign data. Overlapped sign data are presented as chords.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for encoding an audio signal, and more particularly to a technique for encoding an audio signal given as a time-series intensity signal, and decoding and reproducing the audio signal.

[0002]

2. Description of the Related Art As a technique for encoding an audio signal, a PC is used.
The M (Pulse Code Modulation) method is the most widespread method, and is currently widely used as a recording method for audio CDs and DATs. The basic principle of this PCM method is that an analog audio signal is sampled at a predetermined sampling frequency, and the signal strength at each sampling is quantized and represented as digital data. The more it is, the more faithful it is possible to reproduce the original sound. However, the higher the sampling frequency and the number of quantization bits, the larger the required information amount. Therefore, as a technique for reducing the amount of information as much as possible, an ADPCM (Adaptive Differentia) that encodes only a signal change difference is used.
l Pulse Code Modulation) is also used.

[0003] On the other hand, MIDI (Musical Instrume) was born from the idea of encoding musical instrument sounds by electronic musical instruments.
The Digital Interface (nt Digital Interface) standard has also been actively used with the spread of personal computers. Code data according to the MIDI standard (hereinafter, MID)
I data) is basically data that describes the operation of playing a musical instrument, such as which keyboard key of the musical instrument was played and with what strength. The MIDI data itself contains the actual sound. No waveform is included. Therefore, when reproducing the actual sound, the MI which stores the waveform of the musical instrument sound is used.
A DI sound source is required separately. However, the P
Compared to the case where sound is recorded by the CM method, the amount of information is extremely small. This encoding and decoding technology based on the MIDI standard is now widely used in software for playing musical instruments, practicing musical instruments, composing music, etc. using a personal computer, and is also widely used in fields such as karaoke and game sound effects. Have been.

[0004]

As described above, the PC
In the case of encoding an audio signal by the method of M, the amount of information becomes enormous if sufficient sound quality is to be ensured, and the load of data processing must be increased. Therefore, usually
In order to keep the amount of information to a certain extent, we have to compromise on some sound quality. Of course, if the encoding method based on the MIDI standard is adopted, it is possible to reproduce a sound having a sufficient sound quality with a very small amount of information.
Since the I standard itself is originally for encoding the operation of musical instrument performance, it cannot be widely applied to general sound. In other words, in order to create MIDI data, it is necessary to actually play a musical instrument or prepare musical score information.

[0005] As described above, there are advantages and disadvantages in the audio signal encoding method in both the conventional PCM method and the MIDI method, and a small amount of information is sufficient for general audio signals. Sound quality cannot be ensured. However, there is an increasing demand for efficient encoding of general audio signals.

Accordingly, an object of the present invention is to provide an audio signal encoding method capable of efficiently encoding an audio signal.

[0007]

[Means for Solving the Problems]

(1) A first aspect of the present invention provides an audio signal encoding method for encoding an audio signal given as a time-series intensity signal, wherein the audio signal to be encoded is digital audio data. An input stage to be captured, a section setting step of setting a plurality of unit sections on a time axis of the sound data, and a predetermined representative frequency and a representative frequency representing each unit section based on sound data in each unit section Define representative intensity, generate code data including information indicating the start position and end position of each unit section on the time axis and information indicating the representative frequency and the representative intensity, and generate the sound data of each unit section. A coding step of expressing each code data is performed, and further, in the coding step, a section extension process for extending the length of the unit section is performed.

(2) A second aspect of the present invention is the above-mentioned first aspect.
In the audio signal encoding method according to the aspect, in the encoding step, a plurality of tracks having a common time axis are defined, and each unit section set in the section setting step is separated into a plurality of tracks and re-divided. A track separation process to be arranged is performed, and a section extension process is performed for each track.

(3) The third aspect of the present invention is the above-mentioned second aspect.
In the audio signal encoding method according to the aspect, the unit section arranged on a certain track is at least partially overlapped with another unit section arranged on another track on a common time axis. The section extension processing is performed for each track.

(4) The fourth aspect of the present invention is the above-mentioned first aspect.
In the audio signal encoding method according to the third aspect, after performing the unit interval extension process, when unit units having code data similar to each other exist at intervals equal to or less than a predetermined reference interval. In addition, a section integration process for integrating these unit sections into one unit section is performed, and code data of the unit section after integration is determined based on code data of each unit section before integration. .

(5) The fifth aspect of the present invention is the above-mentioned second aspect.
In the audio signal encoding method according to the aspect, when performing track separation processing, adjacent unit sections having code data similar to each other are rearranged on the same track.

(6) The sixth aspect of the present invention is the above-mentioned first aspect.
In the audio signal encoding method according to the fifth to fifth aspects, when performing the unit section extension processing, the maximum section length Lmax and the minimum section length Lmin are set, and each section is set until the adjacent section contacts the adjacent section. The processing is performed to extend up to Lmax at the maximum, and the processing to delete the section whose section length is less than Lmin is performed.

(7) The seventh aspect of the present invention is the above-mentioned first aspect.
In the audio signal encoding method according to the sixth aspect, in the encoding step, the note number is determined based on the representative frequency, the velocity is determined based on the representative intensity, and the delta time is determined based on the length of the unit section. And converts the sound data of one unit section into MIDI-format coded data represented by note number, velocity, and delta time, and assigns different channels to the unit sections overlapping on the time axis. It was made.

(8) An eighth aspect of the present invention is the above-mentioned first aspect.
A program for encoding an audio signal for executing the audio signal encoding method according to the seventh to seventh aspects is recorded on a computer-readable recording medium.

(9) An eighth aspect of the present invention is the above-mentioned first aspect.
Code data encoded by the audio signal encoding method according to the seventh to seventh aspects is recorded on a computer-readable recording medium.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below based on an embodiment shown in the drawings. The present invention is disclosed in Japanese Patent Application No. 9-6746.
Invention disclosed in the specification of No. 7 (hereinafter referred to as the prior invention)
Corresponds to an improved invention based on the basic invention. Therefore, in the following description, the encoding method according to the invention of the prior application will be described first in §1 to §3.

§1. Audio signal encoding according to the prior application invention
Basic Principle of Method First, the basic principle of an audio signal encoding method according to the invention of the prior application will be described with reference to FIG. Now, suppose that an analog sound signal is given as a time-series intensity signal as shown in the upper part of FIG. In the illustrated example, the horizontal axis represents the time axis t, and the vertical axis represents the signal strength A, and the acoustic signal is shown. In the prior application, first, this analog sound signal is
A process for capturing digital audio data is performed. This can be done by using a conventional general PCM technique, sampling the analog audio signal at a predetermined sampling frequency, and converting the signal strength A into digital data using a predetermined quantization bit number. . Here, for convenience of explanation, the waveform of the audio data digitized by the PCM method is also shown by the same waveform as the analog audio signal in the upper part of FIG.

Next, the time axis t of this digital acoustic data
Set multiple unit sections above. In the illustrated example, six unit sections U1 to U6 are set. The position and length of the i-th unit section Ui on the time axis t are indicated by the coordinate values of the start end si and the end ei on the time axis t. For example, the unit section U1 includes a start end s1 to an end e1.
Up to (e1-s1).

When a plurality of unit sections are set in this way, predetermined representative frequencies and representative intensities representing the individual unit sections are defined based on the sound data in each unit section. Here, a state in which the representative frequency Fi and the representative intensity Ai are defined for the i-th unit section Ui is shown. For example, the first unit section U1
, A representative frequency F1 and a representative intensity A1 are defined. The representative frequency F1 is a representative value of the frequency component of the acoustic data included in the section from the start end s1 to the end e1, and the representative intensity Ai is also the start end s1 to the end e1.
Are representative values of the signal intensities of the sound data included in the section up to and including. Generally, the frequency component included in the sound data in the unit section U1 is not single, and the signal strength generally varies. The point of the invention of the prior application is that a single representative frequency and a single representative intensity are defined for one unit section, and encoding is performed using these representative values.

That is, once the representative frequency and the representative intensity are defined for each unit section, information indicating the starting position and the ending position of each unit section on the time axis t, the defined representative frequency and the representative The code data is generated based on the information indicating the intensity, and the sound data of each unit section is expressed by each code data. As a technique for encoding an event that an audio signal having a single frequency and a single signal strength lasts for a predetermined period, encoding based on the MIDI standard can be used. Code data according to the MIDI standard (MI
DI data) can be said to be data expressing sound by musical notes, and FIG. 1 shows the concept of code data finally obtained by musical notes shown in the lower part.

After all, the sound data in each unit section includes pitch information (note number in the MIDI standard) corresponding to the representative frequency F1, intensity information (velocity in the MIDI standard) corresponding to the representative intensity A1, and a unit section. Is converted to coded data having length information (delta time in the MIDI standard) corresponding to the length (e1-s1). The information amount of the code data obtained in this way is significantly smaller than the information amount of the original audio signal, and a remarkable coding efficiency can be obtained. Until now, the only way to generate MIDI data was to perform and encode the operation when the player actually played the instrument, or to input the notes on the musical score as data. By using the technique described above, MIDI data can be directly generated from an actual analog audio signal.

However, there are some points to keep in mind in order to put the above-described coding method into practical use.
The first point is that it is necessary to prepare a sound source during reproduction. Since the code data finally obtained by the above-described method does not include the waveform data of the original sound signal itself, a sound source having some kind of sound waveform data is required. For example, when reproducing MIDI data, a MIDI sound source is required. However,
At present, when the MIDI standard has become widespread, various MIDI sound sources are available, and there is no serious problem in practical use. However, in order to obtain a reproduced sound that is faithful to the original sound signal, it is necessary to prepare a MIDI sound source having waveform data that approximates the sound waveform included in the original sound signal. If reproduction using an appropriate MIDI sound source can be performed, it is possible to obtain a reproduction sound full of a sense of reality with higher sound quality than the original sound signal.

The second point to be noted is that the encoding method is based on the basic principle that the frequency of the sound data included in one unit section is replaced with a single representative frequency. This is unsuitable for encoding an audio signal that includes components at the same time.
Of course, this encoding method can be applied to any audio signal.However, even when encoding an audio signal with a very wide frequency range, such as human voice, Sufficient reproducibility cannot be obtained. To cope with such a problem, a plurality of unit sections may be defined so as to overlap on the time axis, and encoding may be performed separately for each unit section. This technique is described in §6.

A third point to be noted is that in order to perform efficient and highly reproducible encoding, it is necessary to devise a method of setting a unit section. The basic principle of the prior invention is
As described above, the original sound data is divided into a plurality of unit sections, and each unit section is converted into code data indicating a single frequency and a single intensity. Therefore, the finally obtained code data greatly depends on the method of setting the unit section. The simplest method of setting a unit section is a method of uniquely defining unit sections at regular intervals on the time axis, for example, every 10 ms. However, in this method, the unit section is always defined by a constant method regardless of the original audio data to be encoded, and it is not always possible to expect efficient and highly reproducible encoding. Therefore, in practice, it is preferable to analyze the waveform of the original sound data and set a unit section suitable for each sound data.

One approach for efficiently setting a unit section is to extract a section having a similar frequency band from the acoustic data as a single unit section. Since the frequency component in the unit section is replaced by one representative frequency, the reproducibility at the time of reproduction is reduced if a frequency component far away from this representative frequency is included. Therefore, it is important to extract a section in which a frequency approximated to some extent is maintained as one unit section in order to perform efficient coding with good reproducibility. When this approach is adopted, specifically, a change point of the frequency of the original sound data is recognized, and a unit section having the change point as a boundary may be set.

Another approach for efficiently setting a unit section is a method of extracting a section in which signal intensity is approximated from audio data as a single unit section. Since the signal strength in a unit section is replaced by one representative strength, if the signal strength is far from the representative strength, reproducibility at the time of reproduction is reduced. Therefore, extracting a section in which the signal strength approximated to some extent is maintained as one unit section is important for efficient coding with good reproducibility. When this approach is adopted, specifically, a change point of the signal intensity of the original sound data is recognized, and a unit section having the change point as a boundary may be set.

§2. Audio signal encoding according to the prior application invention
Practical Procedure of Method FIG. 2 is a flowchart showing a more practical procedure of the prior application. This procedure includes an input step S10, an inflection point definition step S2
0, a section setting step S30, and an encoding step S40. The input step S10 is a step of taking in an audio signal to be encoded as digital audio data. The inflection point defining step S20 is a step that can be called a preparation step of a later section setting step S30,
In this stage, the inflection point (local peak) is obtained for the waveform of the acquired acoustic data. Also, the section setting step S3
0 is a step of setting a plurality of unit sections on the time axis of the sound data based on the inflection point,
0 is a stage of converting sound data of each unit section into individual code data. The principle of conversion to coded data is as described in §1. That is, based on sound data in each unit section, a predetermined representative frequency and a representative intensity representative of each unit section are defined, and information indicating the start position and the end position of each unit section on the time axis. And information indicating the representative frequency and the representative intensity, code data is generated. Hereinafter, the processing performed in each of these steps will be described in order.

<<< 2.1 Input Stage >>> In the input stage S10, a sampling process S11 and a DC component removing process S12 are performed. Sampling processing S11
Is a process of capturing an analog audio signal to be encoded as digital audio data, and is a process of performing sampling using a conventional general PCM technique. In this embodiment, the sampling frequency is 44.1.
Sampling is performed under the conditions of kHz and the number of quantization bits: 16 bits to prepare digital acoustic data.

Subsequently, the DC component removing process S12 is a digital process for removing a DC component included in the input acoustic data. For example, in the acoustic data shown in FIG. 3, the center level of the amplitude is the center level of the data range indicating the signal strength (specific digital values are, for example, 16
If the data is sampled in bits and a data range of 0 to 65535 is set, the value is 32768. Hereinafter, for convenience of explanation, as shown in the graph of FIG. 3, the center level of the data range is set to 0, and the value of each sampled signal strength is expressed by positive or negative. ing. In other words, this acoustic data includes a DC component corresponding to the value D. If a DC component is included in the analog audio signal subjected to the sampling process, the DC component remains in the digital audio data. Therefore, a process of removing the DC component D is performed by the DC component removal process S12 to make the center level of the amplitude coincide with the center level of the data range. More specifically, a calculation for subtracting the DC component D may be performed so that the average of the individual signal intensities sampled becomes zero. This makes it possible to prepare acoustic data having both positive and negative digital values as signal strength.

<<< 2.2 Inflection Point Definition Stage >>>>
In the inflection point defining step S20, an inflection point search process S21 and a thinning process S22 of the same polarity inflection point are executed. The inflection point search process S21 is a process of finding an inflection point for the waveform of the acquired acoustic data. FIG. 4 is a graph showing a part of the acoustic data shown in FIG. 3 in an enlarged manner with respect to a time axis. In this graph, the points at the tip positions of the arrows P1 to P6 correspond to inflection points (maximum or minimum points), and each inflection point corresponds to a so-called local peak. As a method of searching for such an inflection point, for example, by sequentially paying attention to the sampled digital values along the time axis, it is possible to recognize a position where the value has changed from increasing to decreasing or a position where the value has changed from decreasing to increasing. I just need. Here, this inflection point is indicated by an arrow as shown.

Each inflection point is a point corresponding to one sampled digital data, and has information of a predetermined signal strength (corresponding to the length of an arrow) and information of a position on the time axis t. Will have. FIG. 5 is a diagram showing only the inflection points P1 to P6 indicated by arrows in FIG. In the following description, as shown in FIG. 5, the signal strength (absolute value) of the i-th inflection point Pi is represented by the arrow length a.
i, and the position of the inflection point Pi on the time axis t is ti
Will be shown as After all, the inflection point search processing S21
Is a process for obtaining information on each inflection point as shown in FIG. 5 based on acoustic data as shown in FIG.

The inflection points P1 to P6 shown in FIG.
Has a property that the polarity is alternately inverted. That is, in the example of FIG. 5, the odd-numbered inflection points P1, P3, and P5 are indicated by upward arrows, and the even-numbered inflection points P2, P4, and P6 are displayed.
Is indicated by a downward arrow. This is because the original acoustic data waveform has an original shape as a vibration waveform in which the amplitude alternates between positive and negative. However, actually, such an original vibration waveform is not always obtained. For example, as shown in FIG. 6, a somewhat distorted waveform may be obtained. When the inflection point search processing S21 is performed on the acoustic data as shown in FIG. 6, all of the individual inflection points P1 to P7 are detected, and therefore, as shown in FIG. The orientation does not alternate. However, in defining a single representative frequency, it is preferable to obtain a row of arrows whose directions are alternately inverted.

As shown in FIG. 7, in the thinning process S22 of the same polarity inflection point, when a plurality of inflection points (arrows in the same direction) having the same polarity digital value continue, the digital value having the largest absolute value is obtained. Leaving only the inflection point with the value (the longest arrow)
This is a process of thinning out the rest. In the case of the example shown in FIG. 7, among the three upward arrows P1 to P3, the longest P
2 is left, and only the longest P4 of the three downward arrows P4 to P6 is left.
As a result, as shown in FIG. 8, three inflection points P2, P4,
Only P7 will be left. The inflection point shown in FIG. 8 corresponds to the original shape of the waveform of the acoustic data shown in FIG.

<< 2.3 Section Setting Stage >>>> As described above, in the encoding method according to the prior application,
In order to perform efficient and highly reproducible encoding, it is necessary to devise a method of setting a unit section. In that sense, of the steps shown in FIG. 2, the section setting step S30 is a very important step in practical use. Inflection point definition step S2 described above
0 is a preparation stage of the section setting step S30, and the setting of the unit section is performed using information of each inflection point. That is, in the section setting step S30, processing is performed in accordance with the basic idea of recognizing a change point of the frequency or signal strength of acoustic data based on the inflection point and setting a unit section having the change point as a boundary. Is advanced.

As shown in FIG. 5, the individual inflection points P1 to P6 indicated by arrows have signal intensities a1 to a6, respectively.
6 are defined. However, individual inflection points P1
No information on frequency is defined in P6 itself. The eigenfrequency definition processing S31 performed first in the section setting step S30 includes:
This is a process for defining a predetermined natural frequency. Originally, the frequency is a physical quantity defined for a wave in a predetermined section on the time axis, and should not be defined for a certain point on the time axis. However, here, for the sake of convenience, a pseudo-eigenfrequency is defined for each inflection point (generally, the term “eigenfrequency” in physics refers to an object that vibrates in resonance with a sound wave or the like. Although it means a unique frequency, the “natural frequency” in the present application does not mean such a frequency unique to the object, but a pseudo frequency defined at each individual inflection point, in other words, It means the fundamental frequency at a certain moment of the signal.)

Now, as shown in FIG. 9, among the many inflection points, the nth to (n + 2) th inflection points P (n),
Focus on P (n + 1) and P (n + 2). The signal values a (n), a (n + 1), a
(N + 2) are defined, and positions t (n), t (n + 1), and t (n + 2) on the time axis are defined. Here, considering that each of these inflection points is a point corresponding to the local peak position of the acoustic data waveform,
As shown in the figure, it can be seen that the distance φ on the time axis between the inflection points P (n) and P (n + 2) corresponds to one cycle of the original waveform. Therefore, for example, the n-th inflection point P
The characteristic frequency f (n) of (n) is defined as f (n) = 1.
By defining / φ, a natural frequency can be defined for each inflection point. Position t on the time axis
If (n), t (n + 1) and t (n + 2) are expressed in units of “seconds”, then φ = (t (n + 2) −t (n)), so that f (n) = 1 / (T (n + 2) -t (n)).

In consideration of the actual digital data processing procedure, the position of each inflection point is not represented by the unit of “second” but by the sample number x (the number of data obtained at the sampling time in the sampling processing S11). The sample number x and the actual time “second” are represented by a sampling frequency fs.
Is uniquely associated by For example, the m-th sample x (m) and the (m + 1) -th sample x
The interval on the real time axis between (m + 1) is 1 / fs.

The natural frequency defined at each inflection point in this way is physically an amount indicating a local frequency near the inflection point. If the distance to another adjacent inflection point is short, the local frequency in the vicinity is high, and if the distance to another adjacent inflection point is long,
The local frequency in the vicinity is low.
However, in the above example, the natural frequency is defined based on the distance between the subsequent second inflection point, but any other method may be used to define the natural frequency. I don't care. For example, the natural frequency f (n) of the n-th inflection point is calculated by using the distance between the preceding (n-2) -th inflection point and f (n) = 1 / (t (n) −t (n−2)). Further, as described above, the natural frequency f (n) is calculated as f (n) = 1 / (t (n + 2) -t (n)) based on the distance from the subsequent second inflection point. Even if it is defined by the following formula, since the following two inflection points do not exist for the last two inflection points, f (n) = 1 / ( t (n) −t (n−2)).

Alternatively, the natural frequency f (n) of the n-th inflection point is determined based on the distance from the next succeeding inflection point.
F (n) = (１ ／) · 1 / (t (n + 1) −t
(N)) or f (n) = (3/2) .1 / (t (n + 3)-based on the distance between the following third inflection point. t
(N)) It can also be defined by the following formula. After all, using the general formula, the natural frequency f for the n-th inflection point
F (n) is based on the distance on the time axis between the inflection points separated by k distances (the succeeding inflection point when k is positive, and the preceding inflection point when k is negative). = (K / 2) · 1 / (t (n + k) -t
(N)). The value of k may be set to an appropriate value in advance. If the interval of the inflection point on the time axis is relatively small, setting the value of k to a certain value can define a natural frequency with less error. However, if the value of k is set too large, it is not preferable because the meaning as a local frequency is lost.

When the eigenfrequency definition processing S31 is completed in this manner, the signal intensities a
(N), natural frequency f (n), and position t on the time axis
(N) will be defined.

In §1, in order to perform efficient and highly reproducible encoding, a unit section is set such that the frequency of an inflection point included in one unit section falls within a predetermined approximate range. It has been described that there are a first approach and a second approach in which a unit section is set such that the signal strength of an inflection point included in one unit section falls within a predetermined approximate range. Here, a method of setting a unit section using these two approaches will be described with reference to specific examples.

Now, as shown in FIG. 10, consider the case where signal intensities a1 to a9 and natural frequencies f1 to f9 are defined for each of the nine inflection points P1 to P9. In this case, according to the first approach, it is sufficient to focus on the individual natural frequencies f1 to f9 and perform a process in which a group of spatially continuous inflection points having natural frequencies that are close to each other is set as one unit section. . For example, natural frequencies f1 to f1
f5 has substantially the same value (first reference value), natural frequencies f6 to f9 have substantially the same value (second reference value), and the difference between the first and second reference values. Is outside the predetermined allowable range, as shown in FIG. 10, a section including inflection points P1 to P5 having natural frequencies f1 to f5 included in the approximate range of the first reference value is defined as a unit section U1, The section including the inflection points P6 to P9 having the natural frequencies f6 to f9 included in the approximate range of the second reference value may be set as the unit section U2. In the method according to the invention of the prior application, a single representative frequency is given to one unit section. In this way, a section in which a plurality of inflection points whose natural frequencies are within the approximate range of each other is defined as 1 unit. If it is set as one unit section, the difference between the representative frequency and each natural frequency can be suppressed within a predetermined allowable range, and there is no major problem.

Subsequently, the inflection point at which the natural frequency approximates is 1
An example of a specific method for defining one unit section in a group is shown below. For example, as shown in FIG. 10, when nine inflection points P1 to P9 are given, first, natural frequencies are compared with respect to inflection points P1 and P2, and whether or not the difference between them is within a predetermined allowable range ff Find out what. If | f1−f2 | <ff, the inflection points P1 and P2 are included in the first unit section U1. Then, it is checked whether or not the inflection point P3 may be included in the first unit section U1. This is because the average natural frequency (f1) for the first unit section U1 is
+ F2) / 2 is compared with f3. If | (f1 + f2) / 2−f3 | <ff, the inflection point P3 may be included in the first unit section U1. Further, as for the inflection point P4, if | (f1 + f2 + f3) / 3-f4 | <ff, this can be included in the first unit section U1, and for the inflection point P5, | (f1 + f2 + f3 + f4) / 4 If −f5 | <ff, this can be included in the first unit section U1. Here, if the inflection point P6 is: | (f1 + f2 + f3 + f4 + f5) / 5−f6 |> f
f is obtained, that is, when the difference between the natural frequency f6 and the average natural frequency of the first unit section U1 exceeds a predetermined allowable range ff, the inflection points P5 and P6 And a discontinuous position is detected between the first unit section U1 and the inflection point P6 cannot be included in the first unit section U1. Therefore, the inflection point P5 is the end of the first unit section U1, and the inflection point P6 is another second unit section U2.
And the beginning of Then, the natural frequencies of the inflection points P6 and P7 are compared to determine whether or not the difference between them is within a predetermined allowable range ff. If | f6-f7 | <ff, the inflection points P6 and P7 are P7 is included in the second unit section U2. Then, if | (f6 + f7) / 2−f8 | <ff for the inflection point P8, this is included in the second unit section U2, and | (f6 + f7 + f8) / 3-f9 for the inflection point P9. If | <ff, this is included in the second unit section U2.

By sequentially detecting the discontinuous position by such a method and sequentially setting each unit section, the section setting according to the above-described first approach can be performed.
Of course, the specific method described above is shown as an example, and various other methods can be adopted. For example, instead of comparing with the average value, a simplified method of always comparing the natural frequencies of adjacent inflection points and recognizing a discontinuous position when the difference exceeds the allowable range ff may be adopted. In other words, the individual differences are examined, such as the difference between f1 and f2, the difference between f2 and f3, the difference between f3 and f4, and so on. May be recognized as a discontinuous position.

Although the first approach has been described above, the setting of the unit section based on the second approach can be similarly performed. In this case, the signal intensities a1 to a9 at the individual inflection points may be focused on and compared with the predetermined allowable range aa. Of course, the unit section may be set by combining both the first approach and the second approach. In this case, attention is paid to both the natural frequencies f1 to f9 and the signal intensities a1 to a9 of the individual inflection points. Strict conditions may be imposed, for example, or if one of them falls within the allowable range, a loose condition may be imposed, for example, to include them in the same unit section.

In this section setting step S30, before setting a unit section based on each of the above-described approaches, a process of excluding inflection points having a signal strength whose absolute value is less than a predetermined allowable level is performed. It is preferable to carry out. For example, when a predetermined allowable level LL is set as in the example shown in FIG. 11, the absolute values of the signal intensity a4 at the inflection point P4 and the signal intensity a9 at the inflection point P9 are less than the allowable level LL. In such a case, processing for excluding the inflection points P4 and P9 is performed. The first significance of performing such exclusion processing is to remove noise components included in the original audio signal. Usually, various noise components are often mixed in the process of electrically capturing an audio signal, and it is not preferable to perform encoding including such noise components.

If the allowable level LL is set to a certain level or more, signals other than noise components are also excluded. However, it is sometimes sufficient to exclude signals other than noise components. It becomes processing with. That is, the second significance of performing the exclusion process is to exclude information that is not of interest from information included in the original audio signal. For example, the sound signal shown in the upper part of FIG. 1 is a signal indicating a human heart sound. Among the sound signals, information effective for diagnosing a disease or the like includes a portion having a large amplitude (for each unit section U1 to U6). Part), and the information in the other parts is not very useful. Therefore, when a predetermined allowable level LL is set and a process for excluding unnecessary information portions is performed, more efficient encoding becomes possible.

A component having a relatively small amplitude in a physiological rhythm sound generated by a living body, such as a heart sound or a lung sound,
It is often a reverberation that occurs in the living body, and even if such a reverberation is once removed at the time of encoding, it can be easily added by adding an acoustic effect such as an echo at the time of reproduction. It is possible. In this regard,
The process of excluding inflection points below the acceptable level is significant.

When a process of excluding an inflection point below an allowable level is performed, it is preferable to define a unit section so that division is performed at the position of the excluded inflection point. For example, in the case of the example shown in FIG. 11, unit sections U1 and U2 divided by the positions of the excluded inflection points P4 and P9 (indicated by dashed lines) are defined. If such a unit section definition is made, like the acoustic signal shown in the upper part of FIG.
The section where the signal strength is higher than the allowable level (unit sections U1 to U6)
) And sections below the permissible level (unit sections U1 to U1).
(A section other than U6) alternately appears, so that a very accurate unit section can be defined.

The essential points of the effective section setting method performed in the section setting step S30 have been described above. Here, a more specific procedure will be described. As shown in the flow chart of FIG. 2, this section setting step S30 includes:
It comprises four processes S31 to S34. As described above, the eigenfrequency definition process S31 is a process of defining a predetermined eigenfrequency for each inflection point based on a distance on a time axis between each inflection point and a neighboring inflection point. Here, as shown in FIG. 12, inflection points P1 to P1
Consider an example in which natural frequencies f1 to f17 are defined for each of Nos. 7.

Slicing processing S32 according to the level
Is a process of excluding an inflection point having a signal intensity whose absolute value is less than a predetermined allowable level, and defining a section that is divided at the position of the excluded inflection point. Here, FIG.
Consider the case where allowable levels LL as shown in FIG. 13 are set for the inflection points P1 to P17 as shown in FIG. In this case, the inflection points P1, P2, P11, P16, and P17 are
Inflection points below the acceptable level will be excluded.
In FIG. 14, the inflection points thus excluded are indicated by broken-line arrows. This “slicing process by level S32”
In addition, sections K1 and K2 that are divided at the position of the excluded inflection point are further defined. Here, when there is at least one inflection point excluded, different sections are set to the left and right of the position. As a result, the section K1 from the inflection points P3 to P10 and the inflection point are set. P12-P1
The section K2 up to 5 is set. The sections K1 and K2 defined here are provisional sections, and are not necessarily final unit sections.

The next discontinuous part dividing process S33 searches for a discontinuous position where the value of the natural frequency or the signal strength of the inflection point is discontinuous on the time axis, and separates the individual sections defined in the process S32 into This is a process of defining a new section by further dividing at the discontinuous position. For example, in the case of the above example, provisional sections K1 and K2 as shown in FIG.
Here, if a discontinuity occurs between the inflection points P6 and P7 in the provisional section K1, the provisional section K1 is divided at the discontinuity position and shown in FIG. Thus, provisional sections K1-1 and K1-2 are newly defined, and three provisional sections K1-1, K1-2, and K2 are eventually formed. The specific search method for the discontinuous position is as described above. For example, in the example of FIG. 15, when | (f3 + f4 + f5 + f6) / 4−f7 |> ff, it is recognized that a discontinuity of the natural frequency occurs between the inflection points P6 and P7. Similarly, a discontinuity in the signal strength between the inflection points P6 and P7 is recognized when | (a3 + a4 + a5 + a6) / 4-a7 |> aa.

In the discontinuous part dividing process S33, the conditions for actually performing the section division are as follows. The section is divided only when the natural frequency discontinuity occurs. Performs segmentation only when a natural frequency discontinuity and / or signal strength discontinuity occurs.Either a natural frequency discontinuity and a signal strength discontinuity occur. Various conditions can be set, such as dividing a section only in such a case. Alternatively, in consideration of the degree of discontinuity, it is possible to set a complex condition that combines the above-mentioned conditions.

Thus, in the section obtained by the discontinuous part dividing process S33 (in the above example, three provisional sections K1
-1, K1-2, K2) can be set as the final unit section, but here, the section integration processing S
34. This section integration processing S34 is performed by calculating the average of the natural frequency or signal strength of the inflection point in one section and the natural frequency or signal of the inflection point in the other section in the sections obtained by the discontinuous part division processing S33. When there are two adjacent sections whose difference from the average of the intensity is within a predetermined allowable range, this is a process of integrating the adjacent sections into one section. For example, in the above example,
As shown in FIG. 17, as a result of comparing the sections K1-2 and K2 with the average natural frequency, | (f7 + f8 + f9 + f10) / 4- (f12 + f1
If the difference between the averages is within the predetermined allowable range ff, as in the case of 3 + f14 + f15) / 4 | <ff, the sections K1-2 and K2 are integrated. Of course, the integration may be performed when the difference between the average signal intensities is within the allowable range aa, or when the difference between the average natural frequencies is within the allowable range ff and the difference between the average signal intensities is within the allowable range aa. The integration may be performed when either one of the conditions is satisfied,
Integration may be performed when both conditions are satisfied. Even if such various conditions are satisfied, if the interval between the two sections is more than a predetermined distance on the time axis (for example, a considerable amount of blank space is left because many inflection points are excluded). If there is a section), it is possible to impose a weighting condition not to perform the integration processing.

Thus, the section obtained after performing the section integration processing S34 is set as the final unit section. In the above example, finally, as shown in FIG. 18, the unit section U1 (the provisional section K1-1 in FIG. 17)
And the unit section U2 (the provisional section K1-
2 and K2) are set.

In the embodiment shown here, the starting point and the ending point of the unit section obtained in this way are defined as the starting point at the position on the time axis of the first inflection point included in the section and the last point included in the section. Terminate the position on the time axis of the inflection point,
It is determined by the definition. Therefore, in the example shown in FIG. 18, the unit section U1 is located at the positions t3 to t6 on the time axis.
And the unit section U2 is a position t7 on the time axis.
This is a section from to t15.

<< 2.4 Encoding Step >> Next, the encoding step S40 shown in the flowchart of FIG. 2 will be described. In the embodiment shown here, the encoding step S40 includes a code data generation process S41 and a code data correction process S42. The code data generation processing S41 defines a predetermined representative frequency and a representative intensity representative of each unit section based on the acoustic data in each unit section set in the section setting step S30, and This is a process of generating code data including information indicating a start position and an end position of each unit section, and information indicating a representative frequency and a representative intensity. By this process, audio data of each unit section is converted into an individual code. It will be represented by data. On the other hand, the code data correction process S42 is a process of correcting the generated code data to match the characteristics of the reproduced sound source device used for decoding, as described later.

The specific method of generating the code data in the code data generation processing S41 is very simple. That is, the representative frequency may be defined based on the natural frequency of the inflection point included in each unit section, and the representative intensity may be defined based on the signal strength of the inflection point included in each unit section. This is specifically shown in the example of FIG. This figure 1
8, the unit section U1 including the inflection points P3 to P6
And a unit section U2 including inflection points P7 to P15 (however, P11 is excluded). In the embodiment shown here, the unit section U1 (start end t3, end t3
Regarding 6), as shown in the upper part of FIG. 19, the representative frequency F1 and the representative intensity A1 are calculated by the following formula: F1 = (f3 + f4 + f5 + f6) / 4 A1 = (a3 + a4 + a5 + a6) / 4 t1
Regarding 5), as shown in the lower part of FIG. 19, the representative frequency F2 and the representative intensity A2 are expressed as follows: F2 = (f7 + f8 + f9 + f10 + f12 + f13 +
f14 + f15) / 8 A2 = (a7 + a8 + a9 + a10 + a12 + a13 +
a14 + a15) / 8. In other words, the representative frequency and the representative intensity are simple average values of the natural frequency and the signal intensity of the inflection point included in the unit section. However, as the representative value, not only such a simple average value but also a weighted average value in consideration of the weight may be used. For example, weight individual inflection points based on signal strength,
A weighted average value of the natural frequencies in consideration of the weight may be used as the representative frequency. Alternatively, the maximum value of the signal intensities of the inflection points included in the unit section may be used as the representative intensity.

If the representative frequency and the representative intensity are defined for each unit section in this way, the start position and the end position of each unit section on the time axis have already been obtained. The corresponding code data can be generated. For example, in the case of the example shown in FIG. 18, as shown in FIG. 20, five sections E0, U1, E1, U2, E2
Can be generated.
Here, the sections U1 and U2 are unit sections set in the previous stage, and the sections E0, E1 and E2 are blank sections corresponding to between the unit sections. In each unit section U1, U2,
Although the representative frequencies F1 and F2 and the representative intensities A1 and A2 are respectively defined, the blank sections E0, E1 and E2 are sections in which only the start and end are defined.

FIG. 21 is a table showing an example of the structure of code data corresponding to each section shown in FIG. In this example, the code data shown in one line is a section name (actually,
Unnecessary), the start and end positions of the section, the representative frequency and the representative intensity. on the other hand,
FIG. 22 is a chart showing another example of the structure of the code data corresponding to each section shown in FIG. In the example illustrated in FIG. 21, the start position and the end position of each unit section are directly represented as coded data. However, in the example illustrated in FIG. 22, the information indicating the start position and the end position of each unit section includes the section length. L1 to L4 (see FIG. 20) are used. When the start position and end position of the unit section are directly used as the code data as in the configuration example shown in FIG. 21, the code data for the blank sections E0, E1,. The configuration of FIG. 20 can be reproduced only from the code data of the unit sections U1 and U2 shown in FIG. 21).

The code data finally obtained by the audio signal coding method according to the invention of the prior application is the code data as shown in FIG. 21 or FIG. However,
As the code data, any configuration data may be used as long as it includes information indicating a start position and an end position on the time axis of each unit section and information indicating a representative frequency and a representative intensity. I don't care. As long as the above-mentioned information is included in the finally obtained code data, it is possible to reproduce (decode) a sound using a predetermined sound source. For example,
In the case of the example shown in FIG. 20, silence is maintained during the period from time 0 to t3, a sound corresponding to the frequency F1 is emitted at the intensity A1 during the period from time t3 to t6, and silence is maintained during the period from time t6 to t7. If the sound corresponding to the frequency F2 is played at the intensity A2 during the period from t7 to t15, the original acoustic signal is reproduced.

§3. Using MIDI format code data
That conversion principle to the embodiment <<< 3.1 MIDI data >>>
As described above, the audio signal encoding method according to the prior application invention finally includes information indicating the start position and the end position of each unit section and information indicating the representative frequency and the representative intensity. Any type of code data may be used as long as the code data is the same. However, in practice, such code data is MIDI
Most preferably, code data in a format is adopted. Here, a specific embodiment employing code data in the MIDI format will be described.

FIG. 23 is a diagram showing a configuration of code data in a general MIDI format. As shown, this MID
In the I format, “note on” data or “note off” data exists with “delta time” data interposed. The "delta time" data is composed of data of 1 to 4 bytes and is data indicating a predetermined time interval. On the other hand, "note-on" data is data composed of a total of 3 bytes, the first byte is always fixed to the note-on code "90H" (H indicates a hexadecimal number), and the second byte The code indicating the note number N
A code indicating the velocity V is placed in the third byte. The note number N is a numerical value indicating the number of a musical scale (not a musical scale of seven whole notes in general music, but a musical scale of 12 semitones in this case). A specific keyboard key is designated (the scale of C-2 is associated with the note number N = 0, and 1 to N = 127).
28 scales are associated with each other. (The note A3 at the center of the piano keyboard has a note number N = 69.) The velocity V is a parameter indicating the intensity of the sound (originally, it means the speed of playing the piano keyboard or the like), and defines 128 levels of intensity from V = 0 to 127.

Similarly, the "note-off" data is data composed of a total of three bytes, the first byte is always fixed to the note-off code "80H", and the note number N is stored in the second byte. Is placed in the third byte, and a code representing velocity V is placed in the third byte.
“Note-on” data and “note-off” data are used in pairs. For example, "90 H, 69, 80"
The three-byte "note-on" data means an operation of depressing a key at the center of the keyboard corresponding to note number N = 69, and thereafter, "note-off" data specifying the same note number N = 69 is given. Until the key is depressed, the state in which the key is depressed is maintained (actually, when a waveform of a MIDI sound source such as a piano is used, the waveform of the sound of La is attenuated within a finite time). The “note-off” data specifying the note number N = 69 is given as 3-byte data such as “80H, 69, 50”. For example, in the case of a piano, the value of the velocity V in the “note-off” data is a parameter indicating the speed at which a finger is released from a keyboard key.

In the above description, the note-on code "90H" and the note-off code "80H" are fixed, but the lower 4 bits of these codes are not necessarily fixed to 0. Not channel numbers 0-1
5 can be used as a code to specify any one of the above-mentioned items, and it is possible to specify on / off of the timbre of a different musical instrument for each channel.

As described above, the MIDI data is coded data originally used for describing information related to the operation of the musical instrument performance (in other words, information of the musical score). It is also suitable for use in encoding methods. That is, if the note number N is determined based on the representative frequency F for each unit section, the velocity V is determined based on the representative intensity A, and the delta time T is determined based on the length L of the unit section, It becomes possible to convert the audio data of one unit section into MIDI-format coded data expressed by note number, velocity, and delta time. FIG. 24 shows a specific method of converting to MIDI data.

First, the delta time T of MIDI data
Can be defined by a simple expression of T = L · 768 using the section length L (unit: second) of the unit section. Here, the numerical value “768”
Is based on a quarter note, its length resolution (for example, up to an eighth note if the length resolution is set to 1/2).
/ 8 can express up to thirty-second notes: in general music, the setting of about 1/16 is used), set to the minimum value of 1/384 in the MIDI standard, and set the metronome to four. This is a unique numerical value indicating the time resolution in the MIDI data representation format when the minute note is set to 120 (120 notes per minute).

The MIDI data note number N
In a logarithmic scale where the frequency doubles when the octave goes up, the representative frequency F of the unit section (unit: Hz)
N = (12 / log ₁₀ 2) · (log ₁₀ (F / 44
0) +69. Here, the numerical value “6” of the second term on the right side
"9" indicates the note number (reference note number) of the la sound (A3 sound) at the center of the piano keyboard.
40 Hz), and the numerical value “12” of the first term on the right side
Indicates the scale of one octave when a semitone is counted as one scale.

Further, the velocity V of MIDI data
Using the representative intensity A of the unit section and the maximum value Amax, a value in the range of V = 0 to 127 can be defined by the equation V = (A / Amax) .127. In the case of a normal musical instrument, the velocity V in the “note-on” data and the velocity V in the “note-off” data have different meanings as described above. As the velocity V in the “off” data, the same value as the velocity V in the “note-on” data is used as it is.

In §2 of the previous chapter, audio data in two unit sections U1 and U2 as shown in FIG.
Alternatively, an example in which code data as shown in FIG. 22 is generated has been described, but when MIDI data is used, the unit section U
The audio data in U1 and U2 is represented by data strings as shown in the table of FIG. Here, the note numbers N1 and N2 are values obtained by the above equation using the representative frequencies F1 and F2, and have the velocities V1 and V2.
2 is a value obtained by the above equation using the representative intensities A1 and A2.

<< 3.2 Modification Processing of MIDI Data >>>> Encoding Step S40 in Flowchart shown in FIG.
Then, the code data generation processing S41 is followed by the code data correction processing S42. Code data generation processing S41
Is a process for generating a MIDI data string as shown in FIG. 25, for example, by the above-described specific method. The code data correction processing S42 is a processing for further correcting such a MIDI data string. It is. As will be described later, in order to reproduce (decode) audio based on the MIDI data sequence shown in FIG. 25, a reproduction sound source device (MIDI sound source) having actual audio waveform data is required. The characteristics of the MIDI sound source vary from one sound source to another, and if necessary, it may be preferable to modify the MIDI data in order to match the characteristics of the MIDI sound source to be used. Hereinafter, a specific case in which such correction processing is required will be described.

Now, as shown in the upper part of FIG. 26, audio data in a unit section Ui having a section length Li
Consider a case where the data is represented by DI data (MIDI data before correction). That is, in this unit section Ui, a representative frequency Fi and a representative strength Ai are defined, and a note number Ni, a velocity Vi, and a delta time Ti are set based on the representative frequency Fi, the representative strength Ai, and the section length Li. It will be. At this time, the MID to be used for reproducing the MIDI data
It is assumed that the waveform of the reproduced sound corresponding to the note number Ni of the I sound source is as shown in the middle part of FIG.
In this case, the MIDI is longer than the unit length Li of the unit section Ui.
The duration LLi of the reproduced sound of the sound source is shorter.
Therefore, the MIDI data before correction is
When reproduced as it is using the sound source, the duration LLi is shorter than the duration Li in which the original sound must continue to sound.
Therefore, the reproduced sound is attenuated. When such a situation occurs, the reproducibility of the original sound signal is reduced.

Therefore, in such a case, it is advisable to divide the unit section into a plurality of small sections and perform a correction process for generating separate code data for each small section. In the example shown in FIG. 26, as shown in the lower part of the figure, the original unit section Ui is divided into two small sections Ui1 and Ui2, and separate MIDI data is generated for each section. The representative frequency and the representative intensity defined in each of the small sections Ui1 and Ui2 are the unit sections Ui before the division.
Is the same as the representative frequency Fi and the representative intensity Ai, and only the section length is Li / 2.
MID indicating velocity Vi and delta time Ti / 2
Two sets of I data are obtained.

In a general MIDI sound source, the duration of a reproduced sound is usually determined according to the frequency of the reproduced sound. Especially,
In a sound source for a timbre such as a heart sound, when the frequency of the reproduced sound is f (Hz), the duration is about 5 / f (second). Therefore, when such a sound source is used, for a specific unit section Ui, if the relation between the representative frequency Fi and the section length Li satisfies Li> 5 / Fi, Li / m <5 / Fi. An appropriate number of divisions m such that
It is preferable to perform a process of dividing the unit section Ui into m small sections by the above-described correction processing.

Next, another case that requires a correction process will be described. Now, while the playback sound of the MIDI sound source to be used for playback has a frequency range as shown on the left side of FIG. 27, the frequency range of the playback sound based on the generated series of MIDI data is as shown in FIG. As shown on the right side of 27,
Suppose that a bias is generated by an offset amount d on the bass side. In such a case, the reproduced sound is presented using only a part of the frequency band of the MIDI sound source, which is generally not preferable. Therefore, the average of the frequency of the MIDI data is the center of the frequency range of the MIDI sound source (in this example, the reference tone of 440 Hz (note number N = 6).
9) Perform a correction process to raise the frequency (note number) on the MIDI data as a whole so as to approach
As shown in FIG. 28, the offset amount d is preferably set to zero.

However, depending on the properties of the sound signal, it is preferable to reproduce the sound signal in a state shifted to the lower tone side, and a satisfactory result is not always obtained by the above-described correction processing. Therefore, it is preferable to appropriately determine whether or not to perform such a correction process in consideration of the properties of each acoustic signal.

In addition, depending on the MIDI sound source used, various correction processes may be required in order to match the characteristics. For example, if a MIDI sound source of a special standard that does not correspond to a double frequency of one octave is used, a note number correction process or the like is required to conform to this standard. .

The above-described various processes for encoding are actually performed by computation using a computer, but the computational burden is lighter than computations such as FFT, and a commercially available general-purpose personal computer is used. However, it is possible to sufficiently perform processing in real time. Therefore, if a program for causing a general-purpose personal computer to execute the above-described processing is described, and this program is recorded on a medium such as a floppy disk or a CD-ROM and distributed, the general-purpose personal computer according to the present invention is realized. The present invention can be used as an apparatus for executing an audio signal encoding method. The data encoded by the encoding method according to the present invention can be transferred to a floppy disk or CD-RO by this general-purpose personal computer.
It can also be recorded on a medium such as M and distributed, or transmitted via a communication line.

§4. Basic Concept of Unit Section Extension The present invention is an improvement of the above-mentioned prior application invention, and realizes encoding capable of reproducing with higher quality.
The basic concept is that processing for extending individual unit sections is added to the above-described encoding method. The inventor of the present application has found that the quality at the time of reproduction (at the time of decoding) is improved by extending the unit section of the individual code data obtained by the above-described encoding method on the time axis.

For example, consider a case where code data C1 to C7 as shown in FIG. 29 are obtained by the above-described coding method. Each of the code data C1 to C7 presents a sound corresponding to a predetermined representative frequency with an amplitude intensity corresponding to a predetermined representative intensity during a period from the start point position to the end point position of each unit section U1 to U7. Is shown. According to the experiment performed by the inventor of the present application, it is more difficult to decode and reproduce the code data C1 to C7 as shown in FIG.
Individual unit section U1 indicated by each code data C1 to C7
~ When U7 is extended and decrypted and played back,
Higher quality playback sound was obtained.

One reason for obtaining such a result is that, when the above-described encoding method is performed, the unit section of each piece of coded data finally obtained tends to be shorter than the original length. The inventor of the present application believes that For example, there is a blank section in the gap between the code data C1 to C7 shown in FIG. Such a blank section is
Although the original sound data may have originally been included in the original sound data, the sound data may have been newly generated by the slice processing performed in step S32 of FIG. That is, when the inflection point having the signal strength less than the allowable level is excluded by the slicing process, a new blank section is generated in that part. The extension of the unit section complements the information excluded by the slice processing, and is considered to contribute to the improvement of the quality of reproduced sound.

Another reason for improving the quality of the reproduced sound by performing the extension processing of the unit section is that the human ear recognizes the sound as a sound unless the sound has a certain length or more. It seems to have the property that it cannot do it. For example, the code data C6 in the example shown in FIG. 29 indicates that a predetermined sound is presented over a period corresponding to the unit section U6. Cannot sufficiently recognize the reproduced sound based on the code data C6. When the length of the unit section U6 is extended by the unit section extension processing, such a problem is solved, and it is considered that the quality of the reproduced sound is improved.

However, in the state shown in FIG. 29, extension processing having a sufficient effect cannot be performed. This is because each unit section can only be extended up to a blank section existing at most between adjacent unit sections. Accordingly, the inventor of the present application defines a plurality of tracks having a common time axis, performs a track separation process of separating and relocating individual unit sections set in the section setting stage into a plurality of tracks, and We devised a method to perform the section extension processing for each case. For example, as shown in FIG. 29, unit sections U1 to U7 are set on the time axis t,
When the code data C1 to C7 have been generated, as shown in FIG. 30, the code data (unit section) is separated into two tracks A and B and rearranged. Each of the tracks A and B has a common time axis t, and each code data is rearranged at a position corresponding to the original time axis.

In the illustrated example, odd-numbered code data is arranged on track A, and even-numbered code data is arranged on track B. In this way, when the tracks are separated and rearranged into a plurality of tracks, the blank sections secured between the unit sections are widened, and sufficient section extension processing can be performed. FIG. 31 shows that each of the unit sections U1 to U7 rearranged on the two tracks A and B shown in FIG. 30 is extended until it reaches the maximum section length Lmax, and a new unit section UU1
7 shows an example in which .about.UU7 are formed. By this unit section extension processing, the code data C1 to C7 are replaced with new code data CC1 to CC7. In this replacement, only the section length of each unit section is changed, and the pitch and intensity are changed. There is no. For example, when the code data is MIDI format data, the code data CC1 is the same as the code data C1 in note number and velocity, and only the delta time is changed.

In the example of FIG. 31, it can be seen that the unit section arranged on the track A and the unit section arranged on the track B partially overlap on a common time axis t. For example, in the latter half of the code data CC5, the code data CC5
6 are overlapped, and in this overlap period, the code data CC5 and CC6 are reproduced as chords. Similarly, the second half of the code data CC6 overlaps the first half of the code data CC7, and during this overlap period, the code data CC7 and CC6 are reproduced as chords. By performing separation processing on multiple tracks,
A plurality of code data can be overlapped on the same time axis, and can be reproduced as a chord. Reproduction as such a chord is extremely effective in improving the quality of the reproduced sound. Generally, an audio signal to be encoded originally contains many frequency components. The reason why the sound quality can be improved by performing the reproduction as a chord seems to be that a sound closer to the original acoustic signal can be reproduced by simultaneously reproducing a plurality of frequency components.

In order to further improve the coding efficiency, it is preferable to perform the unit integration process and then perform the unit integration process as necessary. For example,
In the example shown in FIG. 31, code data CC5 and CC7 arranged adjacently on the same track A are code data similar to each other (for example, in the case of MIDI data, note numbers are similar or the same). If the gap d between the two data is less than the predetermined reference interval, the unit sections UU5 and UU7 of each code data are set to 1
It is preferable to perform a process of integrating into one unit section. By integrating the unit sections UU5 and UU7, the code data CC5 and C
Although C7 is also integrated into one piece of code data, the content of the code data after integration may be appropriately determined based on the content of the code data before integration. For example, in the case of MIDI data, the minimum value of the note number and the maximum value of the velocity in the two code data before integration may be set as the value of the code data after integration.

FIG. 32 is a flowchart showing the entire procedure of the encoding method when the above-described unit section extension processing is adopted. Here, the input stage of step S10, step S2
The inflection point defining step of 0 and the section setting step of step S30 are the same as the respective steps of the flowchart shown in FIG. 2, but in the encoding step of step S40, the code data generation processing S41 of step S41 and the coding step S41 of step S42 are performed. A new track separation process in step S50, a unit section extension process in step S60, and a step S
70 section integration processing is added. The processes of steps S50, S60, and S70 may be inserted before the code data generation process of step S41.

The track separation process in step S50 is a process of separating the code data C1 to C7 as shown in FIG. 29 into a plurality of tracks A and B as shown in FIG. The unit section extension processing in the next step S60 is performed by using the code data C for each track as shown in FIG.
In this process, the unit sections 1 to C7 are extended to obtain code data CC1 to CC7 as shown in FIG. If necessary, the section integration processing of step S70 is performed when unit sections having code data similar to each other are arranged adjacently on the same track with an interval equal to or less than a predetermined reference interval. Is integrated into one unit section. In this case, the code data after integration is created based on the individual code data before integration.

§5. Specifically procedure diagram of the unit section extension 33 is a flowchart showing an example of a specific procedure of the track separation process of step S50. In the example shown in FIG. 30, odd-numbered unit sections U1, U3, U5, and U7 are arranged on track A, and even-numbered unit sections U2, U4, and U6 are arranged.
Are arranged on the track B, but when performing the section integration process in step S70, it is preferable to perform track separation by a method that facilitates section integration as much as possible. If track separation processing is performed according to the procedure shown in FIG. 33, track separation is performed such that adjacent unit sections having similar code data are rearranged on the same track, so that section integration is performed in step S70. It will be easier. In the procedure shown in FIG. 33, a general example of separation into a plurality of I tracks is shown.
Here, for the sake of convenience, a case where twelve code data C1 to C12 as shown in FIG. 34 are separated into three tracks A, B and C as shown in FIG. 35 (when I = 3) will be described as an example. . The code data C1 to C1 shown in FIG.
2, it is assumed that any one of N1, N2, and N3 is defined as shown.

First, in step S51, the parameter i is set to an initial value 1. The parameter i indicates the number of the track for which the arrangement processing is being performed. If i = 1, the arrangement processing for the first track A is to be performed. In the following step S52, a process of extracting the leading code data and arranging it on the i-th track is performed. That is, in the case of the example of FIG.
The first code data C1 is arranged on the first track A. In the next step S53, the code data extracted immediately before is searched for code data similar to subsequent code data within a predetermined range. In this embodiment, a condition of “the following code data up to the third” is defined as “the following code data within the predetermined range”. For example, in the case of the example of FIG. 34, the “third succeeding code data” for the code data C1 is code data C2, C3, and C4. In step S53, the three code data C2,
It is searched in C3 and C4 whether there is code data similar to the code data C1. In any case, it can be determined as appropriate whether or not to judge “similar”, but here, when the note numbers match, “similar”
I will decide. In the case of the above example, the code data C
Code data C3 having the same note number as the note number N1 of 1 is searched for as similar code data.

When similar code data is searched, the process branches from step S54 to step S55.
A process of extracting the similar code data and arranging it on the i-th track is performed. In the case of the above example, similar code data C3 is extracted and arranged on the first track A. Then, the process of step S53 is performed again. That is, this time, the code data C extracted immediately before
With respect to No. 3, the presence or absence of similar code data is searched for in the “successive third code data”, and code data C4 is extracted as similar data. Thus, as long as there is similar code data (in this example, code data having the same note number) in the “successive third code data”, the similar code data is extracted and the same code data is extracted. It is arranged one after another on the truck. In the illustrated example, the track A includes code data C1, C
3, C4 are arranged.

However, since there is no code data having the same note number as the code data C4 among the code data C5, C6, and C7 up to the third subsequent to the code data C4, the process proceeds from step S54 to step S56. It will branch. In step S56, a process of extracting predetermined code data and arranging it on the i-th track is performed. In this example, the "predetermined code data" is defined as "the following third code data". Therefore, the third code data C following the code data C4
7 is extracted and placed on track A.

This time, the search in step S53 is performed on the code data C7, and as a result, code data C9 having the same note number is searched. Therefore, steps S54 to S55
Then, a process of extracting the code data C9 and arranging it on the track A is performed. Similarly, the code data C9
Is searched in step S53, and as a result, code data C10 having the same note number
Will be searched. Then, the process branches from step S54 to step S55, and the process of extracting the code data C10 and arranging it on the track A is performed.

Next, a search process in step S53 is performed on the code data C10. In the illustrated example, only two subsequent code data C11 and C12 remain in the code data C10. Before searching for the third code data to be executed, the code data sequence ends. Thus, when the code data sequence ends,
The process branches from step S54 to step S57, and the value of the parameter i is updated by 1. In this example, i = 2
Then, the arrangement processing for the second track B is executed. That is, step S58
Then, the process returns to step S52, and the remaining leading code data C2 is extracted and arranged on the second track B. Thereafter, the arrangement on the track B is performed in the same manner, and as shown in FIG. 35, the code data C5, C6, C
8, C11 will be arranged. Code data C11
Is completed, the code data sequence ends, so the process branches from step S54 to step S57, and the parameter i is updated to 3.

At this point, since i = I, the process branches from step S58 to step S59, and processing for arranging all the remaining code data on the i-th track is performed. In the above example, the last remaining code data C12 is arranged on the third track C.

Thus, the code data C1 to C1 shown in FIG.
C12 has three tracks A, B,
It has been rearranged to C. The point to keep in mind here is
The point is that adjacent unit sections having code data similar to each other (in this example, code data having the same note number) are rearranged on the same track. For example, in the track A shown in FIG.
C3 and C4 are arranged close to each other, and code data C
7, C9 and C10 are also arranged close to each other. In the track B, the code data C2, C5, C6 and C
8, C11 are arranged close to each other. A part of the similar code data arranged in this manner is integrated in the later section integration processing.

FIG. 36 is a diagram showing the result of performing a unit section extension process on the code data separated into the three tracks shown in FIG. 35. In this example, a process is performed in which the maximum section length Lmax is defined as shown in the figure, and each section is extended up to Lmax at maximum until it touches an adjacent section. As a result, the code data C1 to C12 are replaced with the code data CC1 to CC12, respectively. In this example, the code data CC3, CC5, CC9
Cannot be extended to the maximum section length Lmax because all of them contact the following code data, but all other code data are extended to the maximum section length Lmax.

FIG. 37 is a diagram showing the result of performing section integration processing on the code data after the unit section extension processing shown in FIG. That is, when unit sections having similar code data are arranged adjacently on the same track with an interval equal to or less than a predetermined reference interval, a process of integrating these unit sections into one unit section is performed. ing.
Specifically, the code data CC3 and CC4 are the code data C
C34, the code data CC5 and CC6 are integrated into the code data CC56, and the code data CC9 and CC10
Are integrated into the code data CC910. By such section integration processing, the data amount can be compressed, and the coding efficiency can be further improved.

FIG. 38 is a flowchart showing an example of a specific procedure of the unit section extension processing in step S60. The procedure shown here not only extends individual unit sections,
A process of deleting short unit sections that do not satisfy the standard is also performed. That is, first, in step S61, the maximum section length Lmax and the minimum section length Lmin for performing the unit section extension processing are set. The maximum section length Lmax indicates the maximum section length that is allowed to be extended unless it touches an adjacent unit section, and the minimum section length Lmin indicates a criterion for deleting a section whose section length is less than Lmin. This figure 3
In the procedure shown in FIG. 8, a loop process using the parameter Lth is performed so that a process suitable for the significance of the maximum section length Lmax and the minimum section length Lmin is performed. Parameter Lth
Is set to an initial value 0 in step S62,
In step S66, the parameter gradually increases by ΔL. Until it is finally determined in step S64 that Lmin has been reached, steps S63 to S63 are performed.
The loop processing of 66 will be repeated.

First, in step S63, processing is performed to extend each section up to the maximum Lmax until a subsequent section is touched. Unless it is determined in step S64 that the parameter Lth ≧ Lmin, the process proceeds to step S65. If there is a section in which the section length L is less than the parameter Lth, the section is deleted. Such processing is repeatedly executed in step S66 while increasing the value of the parameter Lth by ΔL.

In order to deepen the understanding of the unit section extension processing shown in the procedure of FIG. 38, the track A shown in FIG.
Let's take an example to explain the specific processing contents. In this example, it is assumed that four pieces of code data C1 to C4 are arranged on the track A, and the maximum section length Lmax and the minimum section length Lmin are set to the illustrated length.

First, in step S63, each section is extended. As a result, each code data of the track A is as shown in FIG. That is, the code data C3 and C4 are extended to the maximum section length Lmax and become the code data CC3 and CC4, and the code data C1 and C2 are extended until they contact the following code data and become the code data CC1 and CC2. . In this state,
There is no longer any unit section that can be extended any longer. However, while repeatedly executing the loop from step S63 to step S66, the parameter Lth gradually increases from 0, and the section length of the code data CC2 eventually becomes smaller than the parameter Lth. Then, step S
At 65, the code data CC2 is deleted, and the code data CC2 is deleted from the track A as shown in FIG. Subsequently, by executing step S63 after the deletion processing, the code data CC1 is extended to the maximum section length Lmax, and becomes the code data CCC1 as shown in FIG. 40 (c). Eventually, when the parameter Lth reaches the minimum section length Lmin, the processing procedure ends by branching from step S64. Thus, the processing of deleting the unit section shorter than the minimum section length Lmin has been performed together with the extension processing up to the maximum section length Lmax.

Thereafter, the section integration processing of step S70 is performed. For example, when setting is made such that code data having the same note number and adjacently arranged at intervals equal to or smaller than the reference interval d is to be integrated, FIG.
As shown in (c), code data CCC1 and CC3 which are arranged adjacently at an interval d are subject to integration, and are replaced by new code data CCC13 as shown in FIG. 40 (d). become.

§6. Duplicate unit sections with different representative frequencies
The coding method described so far uses only frequency components limited to some extent in each unit section, such as rhythm sounds generated by living bodies and rhythm sounds generated by nature such as waves and wind. Encoding of the included acoustic signal can ensure practically sufficient reproducibility. However, it is not always possible to ensure sufficient reproducibility when encoding an audio signal that includes very wide frequency components at the same time, such as a human voice called vocal sound. . In particular, it is known that human voice sound has a characteristic called formant (a characteristic in which harmonic components other than harmonics are mixed), and it is difficult to perform encoding with sufficient reproducibility by the above-described method. , Supported by theory. In a general musical instrument, when a certain pitch is played, a frequency component corresponding to the played pitch and a frequency component (an overtone harmonic component) that is an integral multiple of the frequency component are obtained. Therefore, if the performance waveform of such an instrument is used as a MIDI sound source, it is possible to reproduce a sound including harmonic components by the above-described encoding method. However, a human voice having a formant contains harmonic components other than harmonics, so that sufficient reproducibility cannot be ensured.

The method described below has been improved so that it can sufficiently cope with the encoding of a human voice having a formant. FIG. 41 shows the basic concept of this method. In the middle part of FIG. 41, a waveform of digital acoustic data as a time-series intensity signal is shown. Below this waveform, processing focusing on a high frequency is shown, and above this waveform, , The processing focusing on the low frequency band is shown. That is, in the processing focusing on the high frequency shown in the lower half of the figure, the high frequency unit sections Uh (1) to Uh (6) are set, and the representative frequencies Fh (1) to Uh (1) are set for each of these unit sections. Fh (6) and representative strengths Ah (1) to A
h (6) is defined, and high-frequency code data as shown at the bottom of the figure is finally generated. On the other hand, in the processing focusing on the low-band frequencies shown in the upper half of the figure, low-band unit sections Ul (1) to Ul (4) are set, and the representative frequencies Fl (1) to Ul (1) are set for each of these unit sections. Fl (4) and representative strength Al (1) to Al
(4) is defined, and low band code data as shown in the uppermost part of the figure is finally generated.

The important point here is that the high frequency unit section Uh
(1) to Uh (6) and the low frequency unit section Ul (1) to Ul
(4) is that at least a part thereof overlaps on the time axis t. Of course, the time axis t
From the left to the right of the figure, there is a part where only the high-frequency unit section is set, a part where only the low-frequency unit section is set, and also sets any unit section Although there may be a part that is not performed, at least a part on the time axis t includes a section in which the high-frequency unit section and the low-frequency unit section are set to overlap. If the unit intervals set in this way are determined and encoded independently for each representative frequency and representative intensity,
Code data that is duplicated on the time axis is obtained.
For example, in the case of the example shown in FIG. 41, the high-frequency code data shown at the bottom and the low-frequency code data shown at the top overlap at least partially on the time axis t. Sometimes they will be played as chords.
Note that the musical notes shown are for the purpose of illustrating the concept, and are not directly related to the waveforms and the unit sections shown in the middle part of the figure. In this way, if at least partially overlapping unit sections are set on the time axis, and encoding is performed separately for each unit section, a chord including various frequency components can be obtained at the time of reproduction. The sound can be reproduced in the format.

The coding procedure described in §6 can be performed in substantially the same manner as the coding procedure described in §5. That is, as shown in the flowchart of FIG. 2 or FIG. 32, in the input step S10, the audio signal to be encoded is
A process of capturing as digital audio data is performed, and subsequently, in an inflection point defining step S20, a process of obtaining an inflection point for the waveform of the captured audio data is performed.
The processing so far is exactly the same as the procedure already described.
Next, in the section setting step S30, the unit section is set. Here, as described above, the section setting that at least partially overlaps on the time axis is performed. In the encoding step S40, the encoding process is performed for each unit section, and this processing is also performed for each of the unit sections that are set to be overlapped.

The processing first performed in the section setting step S30 is, as described above, the natural frequency definition processing S3.
It is one. At this point, the inflection point search process S21 has already been performed.
In this way, the individual inflection points of the acoustic data waveform are searched, and when a plurality of inflection points having digital values of the same polarity continue by the thinning processing S22 of the same polarity inflection point,
Thinning-out is performed to leave only the inflection point having the largest digital value in absolute value, and the inflection point having a positive signal value and the inflection point having a negative signal value appear alternately.
The eigenfrequency definition process S31 is a process of defining an eigenfrequency for each of such inflection points based on information on the neighborhood. In this method, an eigenfrequency is defined for one inflection point. A plurality of methods are set, and a plurality of natural frequencies are defined at each inflection point by using the plurality of methods.

Here, two specific natural frequency defining methods suitable for use with vocal acoustic signals will be described. Now, let us consider a case where an inflection point group whose part is shown in FIG. 42 is obtained through the inflection point defining step S20. FIG. 42 shows the nth inflection point P (n) to the (n + 12) th inflection point P (n + 12) in this inflection point group. It can be seen that such an inflection point group includes two frequency components. That is, a high frequency component having one cycle of the distance φh between the inflection points P (n) and P (n + 2), and the inflection point P (n)
And a low-frequency component having a period of φ1 between P (n + 6) and P (n + 6) as one cycle. When the inflection point is defined for the vocal acoustic signal, the characteristic as shown in FIG.
This is because, as described above, human voice has the characteristic of formants. In the example shown in FIG.
Inflection points P (n), P (n + 2), P having positive signal strength
Paying attention to (n + 4), P (n + 6), P (n + 8)..., It can be seen that the signal intensity has changed from large, medium, small, large, medium, small,. The period of the change of large, medium and small corresponds to the period φl, and indicates a low frequency component. On the other hand, the appearance period of the inflection point of the same polarity corresponds to the period φh, and indicates a high frequency component.

After all, as a first method of defining a natural frequency for each inflection point, a method of searching for a period φh in which inversion points of the same polarity appear and defining a natural frequency based on this period φh can be adopted. For example, the high frequency eigenfrequency fh can be defined. As a second method of defining a natural frequency for each inflection point, a method of searching for a period φl at which an inflection point having an approximated signal strength appears and defining a natural frequency based on the period φl may be adopted. If the lower natural frequency f
1 can be defined. More specifically, for each inflection point, a specific inflection point that satisfies a predetermined condition is searched for, and a natural frequency may be defined based on a distance on the time axis between the inflection point and the searched inflection point. . For example, FIG.
At the inflection point P (n)
In order to define h, the search may be performed by setting a condition of “the inflection point of the same polarity that appears first thereafter”. As a result, an inflection point P (n + 2) that satisfies this condition is searched for, so that a frequency having a period of the distance φh on the time axis between both inflection points is defined. Similarly, the inflection point P
To define the lower natural frequency fl for (n):
"The inflection point which has a signal strength substantially equal to the signal strength of the inflection point P (n) and subsequently appears first (if the signal strength is given a sign, the inflection point will have the same polarity)." The search may be performed by setting conditions. As a result, an inflection point P (n + 6) that satisfies this condition is searched for, so that a frequency having a period of the distance φl on the time axis between the two inflection points is defined. Thus, by changing the search conditions, it is possible to define a plurality of natural frequencies for the same inflection point.

According to the above method, the n-th inflection point P
The high-band natural frequency fh (n) for (n) is defined in §2.
As described in 3, using an arbitrary integer k, fh (n) = (k / 2) · 1 / (t (n + k) −t
(N)). In other words, a search is made for an inflection point P (n + k) that is k points away from the nth inflection point P (n) (if k is positive, the following inflection point; if k is negative, the preceding inflection point) Position t of the inflection point P (n) on the time axis
Based on the reciprocal of the difference between (n) and the position t (n + k) on the time axis of the searched inflection point P (n + k), the high frequency natural frequency fh (n) is obtained. As described above, if the value of k is set to a relatively large value, a natural frequency with less error can be defined. However, if the value is too large, the meaning as a local frequency is lost.

In the case of the example shown in FIG. 42, the high frequency natural frequency fh (n) for the inflection point P (n) is represented by a period φh
Fh (n) = 1 / φh = 1 / (t (n + 2) −t (n)), which is calculated by the coefficient k in the above equation.
= 2, nothing else. Of course, the coefficient k =
If it is set to 4, the inflection point P (n + 4) is to be searched, and the high-band natural frequency fh (n) is calculated by the following expression: ) Can also be defined.

On the other hand, the low-band natural frequency fl (n) for the n-th inflection point P (n) is obtained by the following equation: fl (n) = 1 / (t (n + k) -t (n)). Will be. However, the coefficient k shown in the denominator on the right side is not an arbitrary integer but an integer satisfying a predetermined condition. That is, the inflection point P (n + k) specified by the integer k is the inflection point (n) of the inflection points having a signal strength within a predetermined error range with respect to the signal strength of the inflection point P (n). Must be the closest inflection point to. Or the integer k
When the preceding inflection point is to be searched, the inflection point P (n + k) specified by the integer k is within a predetermined error range with respect to the signal strength of the inflection point P (n). Of the inflection points having a certain signal intensity, the preceding inflection point closest to the inflection point (n) may be used. The point of this equation is that, in short, the nearest inflection point P (n + k) having the same signal strength as the inflection point P (n) is searched for, and the time axis of the inflection point P (n) is found. Is determined on the basis of the reciprocal of the difference between the position t (n) at the time t and the position t (n + k) on the time axis of the searched inflection point P (n + k). It is.

In the case of the example shown in FIG. 42, the low-band natural frequency fl (n) for the inflection point P (n) is equal to the period φl shown in FIG.
Which is given by the following equation: fl (n) = 1 / φl = 1 / (t (n + 6) −t (n)), which is obtained by calculating the coefficient k in the above equation.
There is nothing other than setting = 6. That is, in the example of FIG. 42, the inflection point P (n + 6) has a signal strength within a predetermined error range with respect to the signal strength of the inflection point P (n), and This means that it has been searched as a near succeeding inflection point. Note that, theoretically, the nearest succeeding inflection point (or the nearest preceding inflection point) does not necessarily need to be a search target. For example, even if the search is made for the second and subsequent inflection point P (n + 12), fl (n) = 2 · (1 / (t (n + 12) −t)
(N))) The low frequency eigenfrequency fl (n) can be defined by the following expression. In general, when the z-th succeeding or preceding inflection point P (n + k) is a search target, fl (n) = Z · (1 / (t (n + k) −t (n))) The low-band natural frequency fl (n) can be defined by the following equation.

Thus, in the case of the present invention, in the natural frequency defining process of step S31 in the flowchart of FIG. 2, a plurality of natural frequencies are defined for each inflection point. The individual processes in steps S32 to S34 are separately performed for a plurality of types of natural frequencies, and the individual processes in steps S41 to S42 are also performed.
It is performed separately for each of a plurality of natural frequencies. As a result, a plurality of code data overlapping on the time axis are generated, and by reproducing these code data on the time axis in a redundant manner, it is also practical for a human voice having a formant characteristic. Reproducibility at various levels can be secured.

For example, in the specific example shown in FIG.
When n = 1 and each inflection point is represented by P1 to P13, if a high-frequency natural frequency is defined for each inflection point,
The natural frequency fhx and the signal strength a as shown in FIG.
An inflection point group having x is defined. If a low-band natural frequency is defined for each inflection point, an inflection point group having a natural frequency flx and a signal strength ax as shown in FIG. 44 is defined. (Where x = 1 to 1)
3). If the two types of inflection point groups are independently and independently subjected to the level-based slicing process in step S32, the discontinuous portion division process in step S33, and the section integration process in step S34, two types of processes are performed. A unit section is set. Here, the unit section set based on the inflection point group having the high frequency eigenfrequency as shown in FIG. 43 is a group of groups in which the high frequency eigenfrequency given to each inflection point falls within a predetermined approximate range. The unit section set based on the inflection point group having the low-frequency natural frequency as shown in FIG. 44 is set as a section including the inflection point, and the low-frequency natural frequency given to each inflection point is The section is set as a section including a group of inflection points that falls within a predetermined approximate range. In short, step S
In the section setting stage of 30, a process of setting a section including a group of inflection points such that the natural frequency defined by the same method falls within a predetermined approximate range is set as one unit section.
The definition of the natural frequency is done in several ways,
A plurality of unit sections overlapping on the time axis are defined.

In the encoding stage of step S40, a representative frequency and a representative intensity are defined independently and independently for each unit section. That is, among the plurality of natural frequencies defined for the inflection point included in the unit section, the representative frequency of the unit section is defined based on the natural frequency involved in setting the unit section, and The representative intensity of the unit section is defined based on the signal intensity of the included inflection point. For example, in the case of the example shown in FIG. 41, for the high-frequency unit section Uh (1), the setting of the unit section of the plurality of natural frequencies defined for the inflection points included in this section Uh (1) The representative frequency Fh (1) is defined on the basis of the high-frequency natural frequency involved in, and the representative intensity Ah (1) is determined based on the signal intensity of the inflection point included in this section Uh (1). Will be defined.

According to the principle of conversion to MIDI data described in §3.1, the MIDI data corresponding to each unit section
The velocity V of the DI data is represented by the representative intensity A of the unit section.
Is normalized by the maximum value Amax and multiplied by 127 to define a formula V = (A / Amax) .127, and a velocity V taking a value of V = 0 to 127 is obtained. When encoding the audio signal, the velocity V is defined by taking the square root of the standardized value and defining the velocity V by an equation of V = (A / Amax) ^1/2 · 127, or by taking the logarithm, = Log (A / Amax) ・ 127 + 127 (However, if V <0, V = 0) It is preferable to define the velocity V because a more natural reproduced sound can be obtained.

Finally, the method described in §6 and §4
An embodiment in which the technique of section extension described in §5 is combined will be described. Now, let us consider a case where acoustic data as shown in FIG. 45 (a) is given. An inflection point is searched for such acoustic data, and a natural frequency and a signal strength are obtained for each inflection point. According to the method described in §6, as shown in FIG. The definition of the low-band natural frequency fl and the definition of the high-band natural frequency fh as shown in FIG. 45 (c) are performed. Then, regarding the unit section set based on the low-band natural frequency, FIG.
The low-frequency code data C11 and C12 as shown in (d) are defined, and the high-frequency code data C21 to C21 as shown in FIG.
C25 is defined.

Subsequently, for each of these code data,
Each unit is extended. However, in the embodiment shown here, the section extension processing is performed on the low-frequency code data shown in FIG. 45D without performing the track separation processing, and the high-frequency code shown in FIG. After the code data is separated into two tracks by the track separation processing, the section extension processing is performed for each track. That is, the code data C21, C23 and C24 of the high band code data shown in FIG.
The data is rearranged on track A as shown in FIG.
2 and C25 are relocated to track B as shown in FIG. Then, code data C2 on track A
1, C23, and C24 are subjected to unit section extension processing, so that code data CC21, C24 as shown in FIG.
C23 and CC24 are obtained, and the code data C22 and C25 on the track B are subjected to the unit section extension processing, whereby the code data CC22 and CC2 as shown in FIG.
5 is obtained. On the other hand, if the unit interval extension processing is performed on the low-frequency code data C11 and C12 shown in FIG.

As a result, as shown in FIG. 47, three tracks A, B, and C having a common time axis t are set, and the code data subjected to the extension processing is arranged in each track separately. Will be done. Therefore, at the time of reproduction, a reproduced sound is presented as a chord of three tracks. However, considering the process of the encoding method, the meanings of the three tracks are slightly different. That is, the code data is separated into the track A and the track B for the convenience of the unit section extension processing, whereas the tracks A, B and the track C are provided based on the high-band natural frequency. This is for arranging the coded data defined in the former, and arranging the coded data defined based on the low-band natural frequency in the latter.

In the above-described embodiment, the track separation processing is not performed on the low-frequency code data for convenience of the unit section extension processing. However, the track separation processing is performed similarly to the high-frequency code data. It doesn't matter. However, in practice, for the low band code data, a sufficient blank section can be secured for the unit section extension processing in many cases. It is.

[0123]

As described above, according to the present invention, efficient coding can be performed on an audio signal.

[Brief description of the drawings]

FIG. 1 is a diagram showing the basic principle of an audio signal encoding method according to the invention of the prior application.

FIG. 2 is a flowchart showing a practical procedure of an audio signal encoding method according to the invention of the prior application.

FIG. 3 is a graph showing digital processing for removing a DC component included in input acoustic data.

4 is a graph showing a part of the acoustic data shown in FIG. 3 in an enlarged manner with respect to a time axis.

FIG. 5 is a diagram showing only inflection points P1 to P6 indicated by arrows in FIG. 4;

FIG. 6 is a graph showing a waveform of acoustic data that has been slightly disturbed;

FIG. 7 is a diagram showing only the inflection points P1 to P7 indicated by arrows in FIG. 6;

8 is a diagram showing a state where a part of the inflection points P1 to P7 shown in FIG. 7 has been thinned out.

FIG. 9 is a diagram illustrating a method of defining a natural frequency for each inflection point.

FIG. 10 is a diagram showing a specific method for setting a unit section based on information about each inflection point.

FIG. 11 is a diagram showing a slice process based on a predetermined allowable level LL.

FIG. 12 is a diagram in which a number of inflection points to be set for a unit section are indicated by arrows.

FIG. 13 is a diagram showing a state in which slicing processing is performed on the inflection point shown in FIG. 12 based on a predetermined allowable level LL.

14 is a diagram showing a state in which inflection points are excluded by the slice processing shown in FIG. 13 and provisional sections K1 and K2 are set.

FIG. 15 is a diagram illustrating a process of searching for a discontinuous position in a provisional section K1 illustrated in FIG. 14;

FIG. 16 is based on the discontinuous position searched in FIG.
The provisional section K1 is divided into new provisional sections K1-1 and K1.
It is a figure which shows the state which defined -2.

17 is a diagram showing an integration process for provisional sections K1-2 and K2 shown in FIG. 16;

18 is a diagram showing unit sections U1 and U2 finally set by the integration processing shown in FIG. 17;

FIG. 19 is a diagram showing a method for obtaining a representative frequency and a representative intensity for each unit section.

FIG. 20 is a diagram showing code data for defining five sections E0, U1, E1, U2, and E2.

FIG. 21 is a table showing an example of code data obtained by encoding sound data in unit sections U1 and U2 shown in FIG. 20;

FIG. 22 is a table showing another example of code data obtained by encoding sound data in unit sections U1 and U2 shown in FIG. 20;

FIG. 23 is a diagram showing a configuration of code data in a general MIDI format.

FIG. 24 is an MID for acoustic data in each unit section.
It is a figure showing the concrete conversion method to I data.

FIG. 25 is a table showing a state in which audio data in unit sections U1 and U2 shown in FIG. 20 are encoded using MIDI data.

FIG. 26 is a diagram illustrating a first case in which generated MIDI data needs to be modified.

FIG. 27 is a diagram illustrating a second case in which the generated MIDI data requires a correction process.

FIG. 28 is a diagram showing a state after correction in the case shown in FIG. 27;

FIG. 29 is a diagram showing an example of code data obtained by the coding method of the invention of the prior application.

30 is a diagram showing a state where the code data shown in FIG. 29 is separated and rearranged into two tracks A and B.

31 extends each of the unit sections U1 to U7 rearranged on the two tracks A and B shown in FIG. 30 until the respective unit sections U1 to U7 reach the maximum section length Lmax, and adds new unit sections UU1 to UU.
It is a figure which shows the example which formed 7.

FIG. 32 is a flowchart showing the entire procedure of an encoding method when a unit section extension process according to the present invention is adopted.

FIG. 33 is a flowchart showing an example of a specific procedure of a track separation process in step S50 shown in FIG. 32;

FIG. 34 is a diagram showing a code data sequence used for describing the procedure shown in FIG. 33.

FIG. 35 is a diagram showing a state in which the code data shown in FIG. 34 is separated and rearranged into three tracks A, B, and C.

36 is a diagram illustrating a state in which unit section extension processing has been performed on code data on each track illustrated in FIG. 35;

FIG. 37 is a diagram showing a state in which section integration processing has been performed on code data on tracks A and B shown in FIG. 36;

FIG. 38 is a flowchart showing an example of a specific procedure of a unit section extension process in step S60 shown in FIG. 32;

FIG. 39 is a diagram showing a code data sequence used for describing the procedure shown in FIG. 38.

40 is a diagram showing a state in which the procedure shown in FIG. 38 has been performed on the code data shown in FIG. 39.

FIG. 41 is a diagram showing a basic principle of an encoding method in which a high-frequency unit section and a low-frequency unit section are respectively defined so that at least a part thereof overlaps on a time axis, and different frequencies are defined in each unit section. It is.

FIG. 42 is a diagram showing a method of defining two types of natural frequencies, a high frequency natural frequency and a low frequency natural frequency, for each inflection point.

FIG. 43 is a diagram showing a state in which a high-band natural frequency and a signal strength are defined for each inflection point shown in FIG. 42;

FIG. 44 is a diagram showing a state where a low-band natural frequency and a signal strength are defined for each inflection point shown in FIG. 42;

FIG. 45 is a conceptual diagram of processing for generating low-frequency code data and high-frequency code data based on the same acoustic data.

FIG. 46 is a diagram showing a state in which the high-frequency code data shown in FIG. 45 (e) is separated into two tracks and rearranged.

FIG. 47 shows the low-frequency code data shown in FIG.
FIG. 7 is a diagram showing a state in which the high-frequency code data 1 and 2 shown in FIG. 6 are separated into separate tracks and subjected to unit section extension processing.

[Explanation of symbols]

A, A1 to A6, Ai: representative intensity Ah (1) to Ah (6): high range representative intensity Al (1) to Al (4): low range representative intensity Amax: maximum value of representative intensity a1 to a13: change Pole signal strength aa Allowable range C1 to C12, C21 to C25 Code data CC1 to CC12, CC21 to CC25, CCC1 Code data after extension processing CC34, CC56, CC910, CCC13 Code data after integration processing D ... DC component d: offset amount / gap between unit sections E0, E1, E2: blank section e1 to e6: terminal position F, F1 to F6, Fi: representative frequency Fh (1) to Fh (6): high-frequency representative frequency Fl (1) to Fl (4): low-frequency representative frequency f1 to f17: natural frequency of inflection point fh1 to fh13: high-frequency natural frequency of inflection point fl11 to fl13: low-frequency natural frequency of inflection point fa, fb, fc: frequency characteristics ff: allowable range fs: sampling frequency K1, K1-1, K1-2, K2: provisional section L, L1 to L4, Li: section length LL: allowable level LLi: reproduction sound continuation Time Lmax ... Maximum section length Lmin ... Minimum section length Lth ... Section length parameters N, N1, N2, N3, Ni ... Note numbers P1 to P17 ... Inflection points s1 to s6 ... Start position T, Ti ... Delta time t1 to t17 Positions on the time axis U1 to U7, Ui, Ui1, Ui2 ... unit sections UU1 to UU6 ... unit sections after extension processing Uh (1) to Uh (6) ... high frequency unit sections Ul (1) to Ul (4) … Low frequency unit section fV, Vi… Velocity x… Sample number φ, φh, φl… Period

Claims (9)

[Claims] 1. An encoding method for encoding an audio signal given as a time-series intensity signal, comprising: inputting an audio signal to be encoded as digital audio data; On the time axis, a section setting step of setting a plurality of unit sections, and based on sound data in each unit section, a predetermined representative frequency and a representative intensity representative of each unit section are defined, Generates code data including information indicating the start position and end position of each of the above unit sections and the information indicating the representative frequency and the representative intensity, and expresses the sound data of each unit section by the individual code data. An audio signal encoding method, comprising: performing an interval extension process to extend a length of a unit interval in the encoding step. 2. The encoding method according to claim 1, wherein in the encoding step, a plurality of tracks having a common time axis are defined, and each unit section set in the section setting step is converted to a plurality of tracks. A sound signal encoding method characterized by performing a track separation process for separating and rearranging, and performing a section extension process for each track. 3. The encoding method according to claim 2, wherein at least a part of a unit section arranged on a certain track is different from another unit section arranged on another track on a common time axis. An audio signal encoding method characterized by performing section extension processing for each track so as to overlap. 4. The encoding method according to claim 1, wherein after performing unit interval extension processing, unit intervals having code data similar to each other are spaced at intervals equal to or less than a predetermined reference interval. If they exist, section integration processing for integrating these unit sections into one unit section is performed, and code data of the unit section after integration is determined based on code data of each unit section before integration. A method for encoding an audio signal, characterized in that: 5. The encoding method according to claim 2, wherein when performing track separation processing, adjacent unit sections having code data similar to each other are rearranged on the same track. Encoding method of the audio signal to be encoded. 6. The encoding method according to claim 1, wherein a maximum section length Lmax and a minimum section length Lmin are set when a unit section extension process is performed, and each section is set to an adjacent section. A method for encoding an acoustic signal, comprising: performing a process of extending a maximum length up to Lmax until a section is touched; and performing a process of deleting a section whose section length is less than Lmin. 7. The encoding method according to claim 1, wherein in the encoding step, a note number is determined based on the representative frequency, a velocity is determined based on the representative intensity, and a length of the unit section is determined. The delta time is determined based on this, and the sound data of one unit section is converted into MIDI-format coded data represented by a note number, a velocity, and a delta time. Is an audio signal encoding method, wherein different channels are assigned. 8. A computer-readable recording medium on which a program for encoding an audio signal for executing the encoding method according to claim 1 is recorded. 9. A computer-readable recording medium on which code data encoded by the encoding method according to claim 1 is recorded.