JP5658506B2 - Acoustic signal conversion apparatus and acoustic signal conversion program - Google Patents

Description

  The present invention relates to an acoustic signal conversion device and an acoustic signal conversion program, and more particularly, to an acoustic signal conversion device and an acoustic signal conversion program for converting a multi-channel acoustic signal into an optimal sound when downmixing.

  A “three-dimensional (three-dimensional) sound method” has been developed that can reproduce sound with a higher presence than the 5.1 channel (ch) surround sound method. Program production using the three-dimensional sound system is performed in a production environment in which a large number of speakers are arranged as standard, but there are many cases where it is not possible to realize sound speaker equipment compatible with the three-dimensional sound system at home. It is reproduced by a 2-channel sound or 5.1-channel surround sound device.

  Therefore, as a technique for reproducing a multi-channel sound signal such as a three-dimensional sound method with a two-channel sound or a 5.1 channel surround sound device, “two-channel sound or "Simult-cast method for transmitting 5.1 channel surround sound signal" and "Downmix method for converting received multi-channel sound signal into 2-channel sound signal, 5.1 channel surround sound signal, etc." There is a prescribed conversion formula for the mix method (for example, see Non-Patent Document 1).

  In recent years, in order to provide better sound to viewers (users), for example, when mixing a narration signal and a background music (hereinafter referred to as “BGM”) signal, the state of mixing balance is changed. A mixing balance display device (for example, see Patent Document 1) that can be displayed and adjusted according to the sense of hearing, or accurately simulates the auditory characteristics of a sound-sensitive deaf person without causing discomfort to the normal hearing person. There is disclosed a mixing balance display system (see, for example, Patent Document 2).

JP 2008-124882 A JP 2009-159083 A

ITU-R Doc 6C / 253-E “PROPOSED PRELIMINARY DRAFT NEW RECOMMENDATION ITU-R BS. [3D-SOUND]”, 26 Oct. 2009

  By the way, when producing an acoustic signal in program production, etc., the balance of speech sounds included in the acoustic signal, such as speech, narration, conversation, speech, etc., and sound effects such as BGM, environmental sound, etc. However, even if the optimum multi-channel sound content is produced in accordance with the intention of the program producer or the sound adjuster, it is possible to use, for example, two-channel sound or When downmixing a 5.1 channel surround sound signal, the utterance sound / sound effect balance deteriorates, for example, it becomes difficult to hear the narration sound due to the BGM sound. There was a case that was not balanced.

  Note that the above-described deterioration of the speech / sound effect balance means, for example, the deterioration of the sound balance (mixing balance) that occurs when two acoustic signals having different signal levels included in the multi-channel acoustic signal are downmixed. Specifically, by performing conventional downmixing, speech / sound effects may be reduced due to differences in spatial acoustic masking conditions depending on the number of playback channels and speaker positions, and unintended signal level increases due to downmixing. It means that the balance deteriorates.

  Therefore, in such a case, it is necessary to separately perform mixing for a 2-channel sound or a 5.1-channel surround sound signal, which is troublesome.

  Conventionally, even when a multi-channel audio signal is downmixed to a 2-channel audio signal, a 5.1-channel surround sound signal, or the like on the receiving side, a technique for improving the deterioration of the speech / sound effect balance as described above. Was not developed.

  The present invention has been made in view of the above-described problems, and in the case of downmixing a multi-channel acoustic signal, an acoustic signal for converting to an optimal sound without degrading the balance between the speech sound and the sound effect. It is an object to provide a conversion device and an acoustic signal conversion program.

  In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

According to the first aspect of the present invention, there is provided an acoustic signal converter for converting an acoustic signal having a first signal level and a second signal level in correspondence with a preset number of channels. The signal level mix balance for the first signal level acoustic signal and the second signal level acoustic signal when the acoustic signal corresponding to the second signal level is downmixed to the acoustic signal corresponding to the second channel number. In accordance with a relative level difference between the first signal level and the second signal level obtained by the balance measuring means , and the first signal level or the second signal. Based on the gain adjustment amount calculating means for calculating the gain adjustment amount of the level and the gain adjustment amount obtained by the gain adjustment amount calculating means, the first signal level is calculated. And / or gain adjustment means for adjusting the gain of the second signal level, and the gain-adjusted acoustic signal obtained by the gain adjustment means, and the first signal level acoustic signal and the second signal level. an acoustic signal by combining the signal levels of, have a synthesizing means for outputting a sound signal corresponding to the number of said second channel, said gain adjustment means, the gain adjustment amount obtained by the gain adjustment amount calculation means Adjusting the gain so that the total volume after synthesis by the synthesis means is kept constant for the set adjustment time. Features.

  According to the first aspect of the present invention, when a multi-channel sound signal is downmixed, it can be converted into an optimal sound without deteriorating the mix balance of two different signal levels.

According to a second aspect of the present invention, there is provided an acoustic signal converter for converting an acoustic signal composed of a first signal level and a second signal level transmitted from a production side in accordance with a preset number of channels. The first signal level when the mixed signal transmitted from the production side is downmixed from the first channel number to the second channel number and the acoustic signal corresponding to the first channel number. And the first signal level or the gain amount of the second signal level corresponding to the mix balance of the second signal level, and the acoustic signal of the first signal level and the second signal level are identified. Mixing metadata separating means for separating the identification metadata for the sound, and an acoustic signal corresponding to the first channel number obtained by the mixing metadata separating means Then, using the identification metadata, the sound signal having the first signal level and the sound signal having the second signal level when the sound signal is downmixed to the sound signal corresponding to the second number of channels. Gain adjustment in which the channel separation means for separating and the gain of the first signal level and / or the second signal level correspond to the relative level difference between the first signal level and the second signal level The first signal level acoustic signal and the second signal level acoustic signal are synthesized using a gain adjustment unit that adjusts based on the amount and a gain-adjusted acoustic signal obtained by the gain adjustment unit. the second and a synthesizing means for outputting a sound signal corresponding to the number of channels, the gain adjustment means, the gain adjustment amount of increase or decrease in the size length tone corresponding to of Set the time, total volume after synthesis by the synthesis unit adjustment time is set and adjusts the gain so as to keep constant.

  According to the second aspect of the present invention, when a multi-channel sound signal is downmixed, it can be converted into an optimal sound without deteriorating the mix balance of two different signal levels.

According to a third aspect of the present invention, the synthesizing unit adjusts the volume of the acoustic signal corresponding to the second number of channels according to the change amount of the acoustic signal gain-adjusted by the gain adjusting unit. It is characterized by.

The invention described in claim 4 is characterized in that the acoustic signal composed of the first signal level and the second signal level is an utterance sound and a sound effect.

According to a fifth aspect of the present invention, an acoustic signal composed of the first signal level and the second signal level to be input is obtained by using the utterance sound or the sound using a preset critical bandwidth of hearing. It is characterized by being separated into sound effects.

The invention described in claim 6 is an acoustic signal conversion program for causing a computer to function as each means included in the acoustic signal converter according to any one of claims 1 to 5 .

According to the sixth aspect of the present invention, when a multi-channel sound signal is downmixed, it can be converted to an optimal sound without deteriorating the mix balance of two different signal levels. In addition, the acoustic signal conversion process can be easily realized by installing the execution program in the computer.

  In addition, what applied the component, expression, or arbitrary combination of the component of this invention to a method, an apparatus, a system, a computer program, a recording medium, a data structure, etc. is also effective as an aspect of this invention.

  According to the present invention, when a multi-channel sound signal is downmixed, the sound can be converted to an optimum sound without deteriorating the mix balance of two different signal levels.

It is a figure which shows an example of the acoustic signal conversion system in 1st Embodiment. It is a figure which shows the example of arrangement | positioning of the acoustic speaker at the time of 22.2 channel. It is a figure which shows an example of the calculation formula at the time of downmixing from the acoustic signal of 22.2 channel. It is a figure for demonstrating the example of gain adjustment amount calculation in this embodiment. It is a figure which shows an example of the acoustic signal conversion system (transmission side) in 2nd Embodiment. It is a figure which shows an example of the acoustic signal conversion system (reception side) in 2nd Embodiment. It is a figure which shows an example of the relationship between a critical band number and a frequency. It is a flowchart which shows an example of the acoustic signal conversion process sequence in 1st Embodiment. It is a sequence diagram which shows an example of the acoustic signal conversion process procedure in 2nd Embodiment.

<About the present invention>
In the present invention, for example, in parallel with the production of multi-channel sound content having a large number of sound channels (first channel number) such as a three-dimensional sound system, a multi-channel sound signal content is converted into a two-channel sound signal or It is automatically converted (downmixed) into audio signal content of the number of channels (second channel number) such as audio signals of 1-channel surround content. In the present invention, the program content having a large number of audio channels and the metadata generated in association with the program content are used, and on the receiving side, the 2-channel audio signal and 5.1 channel surround are based on the above-mentioned metadata. It converts (downmix) into an acoustic signal corresponding to the receiving side acoustic equipment, such as an acoustic signal. The present invention has a function for adjusting the balance between the utterance sound and the sound effect in the above-described conversion of the acoustic signal.

  Next, an embodiment in which an acoustic signal conversion device and an acoustic signal conversion program according to the present invention having the above-described features are suitably implemented will be described in detail with reference to the drawings. In the following embodiments, when the production side produces a two-channel acoustic signal by downmixing simultaneously with the multichannel acoustic signal, and the reception side produces the two-channel acoustic signal by downmixing from the multichannel acoustic signal. The case of producing will be described. In the present embodiment, a 2-channel acoustic signal described below can be easily replaced with an acoustic signal having a different number of channels (for example, a 5.1 channel surround acoustic signal).

<Acoustic Signal Conversion System: First Embodiment>
FIG. 1 is a diagram illustrating an example of an acoustic signal conversion system according to the first embodiment. An acoustic signal conversion system 10 shown in FIG. 1 is configured to include an acoustic recording / reproducing device 11, a microphone 12 as an audio input means, an acoustic mixing device 13, and an acoustic signal conversion device 14. The acoustic signal conversion device 14 includes a speech sound / sound effect balance measurement device 21, a gain adjustment amount calculation unit 22, a gain adjustment unit 23, and a synthesis unit 24.

  The acoustic signal conversion system 10 shown in FIG. 1 is an example of a two-channel audio signal automatic downmix technique that is performed in parallel with the production of acoustic content having a large number of acoustic channels on the production side.

  In the acoustic signal conversion system 10 shown in FIG. 1, the sound recording / reproducing device 11 is configured to store the sound signals such as narration and speech recorded in advance and the sound signals such as sound effects such as BGM and the like. An acoustic signal to which attribute data “speech sound / sound effect identification” set in advance in order to grasp the contents is added is generated.

  Further, the microphone 12 directly inputs sound effects such as narration such as an announcer, speech, etc. and environmental sounds, etc., and the above-described attribute data “speech sound / sound effect identification” is added to each input. Generate a signal. That is, the sound source material obtained from the sound recording / playback apparatus 11 and the microphone 12 is output with the attribute “speech sound / sound effect identification” relating to the semantic content of the sound signal added for each sound signal.

  Here, “speech sound / sound effect identification” as attribute data is, for example, identification information indicating whether the sound content of the channel is “speech sound” or “sound effect” for each sound channel signal. . In the case of the first embodiment, the sound recording / reproducing device 11 and the microphone 12 normally receive either one of “speech sound” or “sound effect”, so that the input sound signal Corresponding identification information set in advance can be added.

  For example, when an acoustic signal corresponding to “sound effect” is input to the sound recording / playback apparatus 11 and an acoustic signal corresponding to “utterance sound” is input to the microphone 12, attribute data of identification information corresponding to each is input. Will be added. In the present embodiment, the present invention is not limited to this. For example, an audio signal of “speech sound” may be input to the sound recording / playback apparatus 11, and an audio signal of “sound effect” may be input to the microphone 12. May be. In the present embodiment, each of the sound recording / reproducing device 11 and the microphone 12 has one or a plurality of microphones. Therefore, one identification information corresponding to the sound signal is added to each input, and the sound mixing device 13 is added. Is output.

  The acoustic mixing device 13 receives one or a plurality of sound signals (sound source material) from one or a plurality of sound recording / playback devices 11 or microphones 12, and uses a plurality of sound channels such as a three-dimensional sound system using the input sound signals. In order to generate the audio content having the above, mixing is performed according to conditions set in advance by the audio adjuster, and the audio content corresponding to a large number of target audio channels (for example, 22.2 channels) is generated. The acoustic mixing device 13 outputs the generated multi-channel acoustic signal content 31.

  In the example of FIG. 1, the multi-channel audio signal content 31 produced by the audio mixing device 13 adjusts the balance of the utterance sound / sound effect according to the intention of the program producer set by the genre of the program, for example. The optimal acoustic signal is produced by adjustment by the person.

  Also, the acoustic mixing device 13 classifies the produced multi-channel acoustic signal content 31 for each utterance sound and sound effect using the above-described attribute data “speech sound / sound effect identification”, and the utterance sound and effect. Using a conversion formula or the like set in advance for each sound, down-mixing (sound signal conversion) is performed on the two-channel sound signal, and a speech sound down-mix signal 32 and a sound effect down-mix signal 33 are output.

  Further, as the above-described conversion formula, for example, a specified conversion formula as shown in Non-Patent Document 1 described above can be used. However, the present invention is not limited to this conversion formula. Other conversion formulas may be used depending on the environment of the sound equipment on the side (space, performance of speakers that output sound, etc.), the person who listens to the sound (user), the deaf or the elderly, etc. it can. A specific example of downmix using a conversion formula will be described later.

  As described above, the audio mixing device 13 is produced based on the intention of the program producer and the mixing information operated by the trained acoustic adjuster with the optimum utterance / sound effect balance in accordance with the intention of the program producer. The multi-channel sound content 31 is output, and the above-described two-channel downmix signals (speech sound downmix signal 32 and sound effect downmix signal 33) are output.

  In the acoustic signal conversion device 14, the utterance sound / sound effect balance measurement device 21 receives the utterance sound downmix signal 32 and the sound effect downmix signal 33 and mixes them as described in, for example, Patent Document 1 and Patent Document 2 described above. Input to a balance display device or display system, and measure the balance of speech / sound effects.

  Specifically, as shown in Patent Document 1, the levels of the first and second sound signals (the speech sound downmix signal 32 and the sound effect downmix signal 33) are detected for each frame of a predetermined time interval, A level difference between the first sound signal and the second sound signal is calculated, a weighted level difference is calculated by weighting the level difference according to the level of the first sound signal, and the past difference from the current frame is calculated. The average value of m values is calculated in descending order of the weighted level difference value up to n frames, and the level of the first sound signal from the current frame to a predetermined number of frames in the past An average value is calculated, and a display value indicating a state of mixing balance between the first sound signal and the second sound signal is determined based on each calculation result.

  Further, as shown in Patent Document 2, the energy levels of the first and second sound signals (the speech sound downmix signal 32 and the sound effect downmix signal 33) are calculated for each frequency band, and the first sound signal Based on the difference between the energy level and the energy level of the second sound signal, a masking correction amount for simulating the auditory masking characteristics of the hearing-impaired deaf person is calculated for each frequency band, and the energy level of the first sound signal is calculated. Based on the sum of the energy level of the second sound signal and the second sound signal, a recruitment correction amount for simulating the recruitment phenomenon of the sound-sensitive deaf person is calculated for each frequency band, and the masking correction amount and the recruitment correction amount are calculated. Based on the first and second sound signals, first and second auditory characteristic simulation signals simulating the auditory characteristics of the sound-sensitive deaf person are calculated.

  Further, the utterance sound / sound effect balance measuring device 21 outputs the balance measurement result (signal level or the like) of each utterance sound / sound effect to the gain adjustment amount calculation means 22.

  The gain adjustment amount calculation means 22 calculates a gain adjustment amount for the level of the utterance sound downmix signal 32 based on the measurement result of the utterance sound / effect sound balance obtained from the utterance sound / effect sound balance measurement device 21. However, the present invention is not limited to this, and the gain adjustment amount calculation means 22 calculates the gain adjustment amount with respect to the level of the sound effect downmix signal 33 based on, for example, the measurement result of the utterance sound / effect sound balance. It may be calculated. Further, the gain adjustment amount may be calculated for the signal levels of both the utterance sound downmix signal 32 and the sound effect downmix signal 33 based on the measurement result of the utterance sound / effect sound balance. A method for calculating the gain adjustment amount in the present embodiment will be described later. The gain adjustment amount calculation unit 22 outputs the obtained gain adjustment amount to the gain adjustment unit 23.

  The gain adjusting unit 23 performs gain adjustment on the speech sound downmix signal 32 based on the gain adjustment amount obtained by the gain adjustment amount calculating unit 22. In the present invention, the present invention is not limited to this. For example, the gain adjustment unit 23 calculates the gain adjustment amount with respect to the level of the sound effect downmix signal 33 by the gain adjustment amount calculation unit 22. Gain adjustment is performed on the sound effect downmix signal 33. Furthermore, if the gain adjustment means 23 has calculated the gain adjustment amount for the signal levels of both the utterance sound downmix signal 32 and the sound effect downmix signal 33 by the gain adjustment amount calculation means 22, the utterance sound. Gain adjustment is performed on the downmix signal 32 and the sound effect downmix signal 33.

  Furthermore, the gain adjusting means 23 prevents the gain adjustment value from abruptly changing, for example, within a program or within a certain period of time in order to maintain temporal continuity by adjusting the gain of the speech sound downmix signal 32. Specifically, the gain adjustment unit 23 sets an adjustment time according to the gain adjustment amount. For example, when the gain adjustment amount increases or decreases, the gain adjustment unit 23 sets the adjustment time longer according to the amount. And adjust so that there is no sudden fluctuation over time.

  The synthesizing unit 24 synthesizes the utterance sound downmix signal 32 gain-adjusted by the gain adjusting unit 23 and the sound effect downmix signal 33, so that the balance of the utterance sound / sound effect in accordance with the intention of the program producer. 2 channel acoustic signal content 34 is output. Note that one or both of the speech sound downmix signal 32 and the sound effect downmix signal 33 synthesized by the synthesizing unit 24 may be gain-adjusted.

  Here, in the embodiment shown in FIG. 1, when the gain of the speech sound downmix signal 32 is adjusted, if the amount of adjustment is large, the level of the downmix sound synthesized by the synthesizing unit 24 increases or decreases. May end up. Therefore, for example, the synthesizing unit 24 in the present embodiment keeps the total volume after synthesis, for example, substantially constant in accordance with the amount of change in the acoustic signal (speech sound downmix signal 32) gain-adjusted by the gain adjusting unit 23. As described above, the volume is adjusted before or after the synthesis of the speech sound downmix signal 32 and the sound effect downmix signal 33. For example, when the signal level is raised by adjusting the gain of the utterance sound downmix signal 32, the synthesizing unit 24 lowers the signal level of the sound effect downmix signal 33 before synthesis corresponding to the rate of increase. Alternatively, the signal level of the two-channel sound signal after synthesis may be lowered so that the total volume becomes substantially constant.

  According to the first embodiment described above, it is possible to automatically produce a sound source material (or multichannel audio content) for producing multichannel audio content by downmixing. Further, according to the first embodiment, the production side can produce a 2-channel audio signal by downmixing in parallel with the multi-channel audio signal.

  Therefore, the multi-channel audio signal content 31 and the 2-channel audio signal content 34 obtained in the first embodiment can be used for different purposes, respectively, and the multi-channel audio signal content 31 and the 2-channel audio signal content 34 can be used. Can be transmitted simultaneously to provide the user with audio signal content as a simulcast method.

  According to the first embodiment described above, in the automatic downmix on the production side, an appropriate utterance / sound effect balance is performed simultaneously with the production of the audio content having a large number of audio channels such as a three-dimensional audio system. Audio content such as 2-channel sound and 5.1-channel surround sound in which sound quality is maintained can be automatically downmixed. Further, in accordance with the prescribed downmix calculation formula and the utterance sound / sound effect identification signal regarding the meaning content of the sound signal, the utterance sound downmix signal and the sound effect downmix signal are generated, and the utterance sound signal and Measures the sound / sound effect balance from the sound signal, adjusts the level of the sound signal based on the result of the sound / sound balance measurement, and synthesizes the gain adjusted sound signal and sound effect signal. Thus, it is possible to produce a downmix signal in which an appropriate utterance / effect sound balance is maintained.

<Specific example of downmix using conversion formula>
Here, a specific example of the downmix using the conversion formula described above will be described with reference to the drawings. In the following description, 22.2 channel sound is used as an example of multi-channel sound, but the present invention is not limited to this.

  FIG. 2 is a diagram illustrating an arrangement example of acoustic speakers (sound systems) in 22.2 channel. FIG. 3 is a diagram illustrating an example of a calculation formula used when downmixing from 22.2 channel acoustic signals.

  3A shows an example of a downmix type from the 22.2 channel to 2 channels and a bass sound channel, and FIG. 3B shows a downmix type from the 22.2 channel to 5.1ch. And an example of a bass acoustic channel.

  For example, in the 22.2 channel, as shown in FIG. 2, in the arrangement of the speakers in the three-dimensional space with respect to the TV screen (TV Screen), the top layer has 9 channels (TpFL, TpFC, TpFR, TpSiL, TpC, TpSiR, TpBL). , TpBC, TpBR), 10 channels (FL, FLc, FC, FRc, FR, SiR, BR, BC, BL, SiL) in the middle layer, 3 channels (BtFL, BtFC, BtFR), LFE (Low Frequency) in the bottom layer (Effect: low-frequency effect sound) has two channels (LFE1, LFE2).

  In this case, in the case of downmixing from 22.2 channels to 2 channels, for example, by using the equations (1) and (2) shown in FIG. ) Acoustic signal can be calculated.

  Note that according to the present embodiment, it is possible to similarly downmix to other channels. For example, in the case of downmixing from 22.2 channels to 5.1 channels, as shown in FIG. ) To (7) to calculate 5.1 channel (L, R, C, LS, RS, LFE) acoustic signals which are base acoustic channels.

  In addition, the downmix type as shown in FIG. 3 is shown by the nonpatent literature 1 etc. which were mentioned above, for example.

<Example of the number of channels applicable to this embodiment>
Here, the acoustic signal that can be input to perform the acoustic signal conversion (downmix) in the present embodiment described above is not limited to the 22.2 channel described above. Two channels, 9.1 channels, 8.1 channels, 7.1 channels, 6.1 channels, etc. can be used.

  Also, the number of channels to be downmixed preferably corresponds to, for example, the number of channels of audio equipment that can be realized in a general home. For example, the above-described 2 channels or 5.1 channels, and further, 1 channel or 3 channels It can also be applied to 5 channels (without LFE).

<Calculation example of gain adjustment amount>
Next, a gain adjustment amount calculation example in the gain adjustment amount calculation means 22 will be described with reference to the drawings. FIG. 4 is a diagram for explaining a gain adjustment amount calculation example in the present embodiment. FIG. 4 shows an example of the gain adjustment value function of the level of the utterance sound signal (utterance sound downmix signal 32).

  As shown in FIG. 4, when the horizontal axis is a weighted relative level difference (“sound effect” − “utterance sound”) and the vertical axis is a gain adjustment amount (dB), for example, the weighted relative level difference is “ The gain adjustment is performed so that the gain adjustment amount (dB) increases linearly from “0” to “6” as it increases from “−6” to “0”. As described above, by using a preset gain adjustment value function, the corresponding adjustment amount can be easily calculated based on the measurement result of the utterance sound / sound effect balance.

  The example shown in FIG. 4 shows an example of the gain adjustment value function of the speech signal level. However, the present invention is not limited to this, and for example, a sound effect signal (sound effect downmix signal 33). ) A level gain adjustment value function may be set to adjust the sound effect signal level, or both the speech sound signal level and the sound effect signal level may be adjusted using the above-described function.

  The function shown in FIG. 4 is set for each utterance sound signal level and sound effect signal level, and by setting another function according to the number of channels to be downmixed, Optimal level adjustment can be realized according to the type and number of channels.

  In the present embodiment, the level increase of the downmix sound accompanying the gain adjustment of the utterance sound signal is, for example, after the synthesis of the utterance sound signal and the sound effect signal (after the downmix) or so as to maintain the total volume. Adjust before synthesis. Also, in order to maintain temporal continuity by adjusting the gain of the utterance sound signal, the gain adjustment value is not suddenly changed within the program or within a certain time.

<Acoustic Signal Conversion System: Second Embodiment>
Next, a second embodiment of the acoustic signal conversion system will be described with reference to the drawings.
FIG. 5 is a diagram illustrating an example of an acoustic signal conversion system (transmission side) in the second embodiment. FIG. 6 is a diagram illustrating an example of an acoustic signal conversion system (reception side) in the second embodiment. That is, the acoustic signal conversion system 40 according to the second embodiment is roughly classified into a transmission-side acoustic signal conversion system 40-1 shown in FIG. 5 and a reception-side acoustic signal conversion system 40-2 shown in FIG. The

  In the second embodiment, in order to generate a two-channel sound signal by downmixing from a multi-channel sound signal in the sound signal conversion device and provide it to the user, the sound signal production device on the transmission side generates mixing metadata, An example is shown in which a receiving-side acoustic signal conversion device receives mixing metadata transmitted from an acoustic signal production device and generates a two-channel acoustic downmix signal using the mixing metadata.

  In the following description, the same functional configuration in the above-described first embodiment shown in FIG. 1 is denoted by the same reference numeral, and a specific description thereof is omitted here.

<Configuration of transmission side of acoustic signal conversion system>
The acoustic signal conversion system 40-1 shown in FIG. 5 is configured to include an acoustic recording / reproducing device 11, a microphone 12 as an audio input unit, an acoustic mixing device 43, and an acoustic signal production device 44. The acoustic signal production device 44 includes an utterance sound / sound effect balance measurement device 21, a gain adjustment amount calculation unit 22, and a mixing metadata multiplexing unit 45.

  In the transmission-side acoustic signal conversion system 40-1 shown in FIG. 5, as shown in the first embodiment described above, one or more acoustic signals output from one or more acoustic recording / reproducing devices 11 and the microphone 12. (Sound source material) is input by the acoustic mixing device 43. In addition, as described above, “speech sound / sound effect identification”, which is attribute data related to the meaning content of the sound signal, is added in advance to the sound source material for multi-channel sound content production in advance for each sound signal to be input. Has been.

  The sound mixing device 43 creates sound content of multi-channel sound having an optimal utterance / effect sound balance in accordance with the intention of the program producer. In other words, at the time of production, based on the mixing information operated by the sound adjuster trained according to the program producer's intention, etc., the optimal utterance / sound effect balance in accordance with the program producer's intention, etc. The produced multi-channel sound content is produced.

  Also, the acoustic mixing device 43 outputs the multi-channel acoustic signal content 31, the utterance sound downmix signal 32, and the sound effect downmix signal 33 in the same manner as in the first embodiment described above, and further utterance sound / sound effect identification. The metadata 51 is output. That is, in the second embodiment, the multi-channel sound signal produced by the sound mixing device 43 includes the utterance sound / sound effect identification metadata 51 for each channel sound signal based on the mixing information operated by the sound adjuster. Is generated.

  More specifically, the utterance sound gain metadata and the utterance sound / sound effect identification metadata 51 attached to each channel sound signal are first set to “speech sound / sound effect identification” of attribute data added in advance. Utterance sound / sound effect identification metadata 51 for each set acoustic channel signal is generated based on the mixing information operated by the sound adjuster. When the acoustic content of the channel signal includes both “speech sound” and “sound effect”, the frequency band is divided into a plurality of frequency bands having a critical bandwidth of hearing, for example, "Speech sound / sound effect identification" can be added for each signal. An additional example of “speech sound / sound effect identification” will be described later.

  Next, the acoustic mixing device 43 performs two types of downmix signals, a two-channel sound downmix of the utterance sound and a two-channel sound downmix of the sound effect, according to the attribute data “speech sound / sound effect identification” added in advance. Are generated in accordance with the mixing information operated by the sound adjuster and the prescribed downmix calculation formula as shown in FIGS.

  The utterance sound / sound effect balance measurement device 21 of the sound signal production device 44 performs balance measurement based on the input utterance sound downmix signal 32 and the sound effect downmix signal 33. In the second embodiment, the speech sound signal and the sound effect signal are input to, for example, a mixing balance display device disclosed in Patent Document 1 or Patent Document 2, and the balance of the speech sound / sound effect is measured.

  Further, the gain adjustment amount calculation means 22 is for a two-channel sound downmix signal for gain adjustment of the level of the utterance sound signal, for example, as described above based on the input utterance sound / effect sound balance measurement result. The gain adjustment amount 52 consisting of the utterance sound gain metadata is calculated.

  Here, the “speech gain metadata” includes, for example, the output value of the function of the gain adjustment value of the speech sound signal level based on the result of the balance measurement of the speech sound / effect sound, as in FIG. 4 described above. .

  In the second embodiment, as in the first embodiment described above, the gain adjustment amount 52 for the signal level of the sound effect downmix signal 33 is calculated, or the speech sound downmix signal 32 and the sound effect downmix are calculated. The gain adjustment amount 52 for the signal level of the signal 33 may be calculated.

  Therefore, in the second embodiment, the gain adjustment amount calculation unit 22 generates and outputs a gain adjustment amount 52 as, for example, speech sound gain metadata associated with one sound channel.

  In addition, the gain adjustment value in the second embodiment has a plurality of gains so that, for example, the receiving side can change the level according to the conditions of each user, such as a hearing impaired person, an elderly person, and a sound preference for each user. An adjustment value may be included. By including a plurality of gain adjustment values in this way, the user or the like can select the sound that suits him / her on the receiving side.

  The mixing metadata multiplexing unit 45 uses, as the mixing metadata for the multi-channel sound signal content 31, the speech sound / sound effect identification metadata 51 for each channel sound signal, and one gain adjustment amount 52 for all sound channels. Then, the multiplexed mixing signal is transmitted to the acoustic signal converter 60 side.

  The timing at which the speech / sound effect identification metadata 51 and the gain adjustment amount 52 are multiplexed on the multi-channel audio signal content 31 by the mixing metadata multiplexing unit 45 may be, for example, every preset time interval. preferable. Thereby, for example, even when the receiving side switches to another program in the middle of the program, it is possible to provide the user with the optimal sound adjusted quickly. In the present invention, the present invention is not limited to this. For example, it may be performed every time a plurality of programs (sound contents) are switched, or may be when there is an acoustic adjustment request from the receiving side.

<About the configuration on the receiving side of the acoustic signal conversion system>
Next, the receiving side that receives the mixing signal transmitted from the acoustic signal production device 44 of the acoustic signal conversion system 40-1 using FIG. 6 and outputs the received signal using acoustic output means such as a speaker is used. A functional configuration of the acoustic signal conversion device 60 which is the acoustic signal conversion system 40-2 will be described with reference to the drawings.

  The acoustic signal conversion device 60 shown in FIG. 6 is configured to include mixing metadata separation means 61, channel separation means 62, gain adjustment means 63, and synthesis means 64. Note that the acoustic signal converter 60 shown in FIG. 6 outputs a two-channel acoustic signal 71. Note that the present invention is not limited to this. For example, the 5.1 channel acoustic signal described above may be output.

  When the acoustic signal conversion device 60 receives the mixing signal transmitted from the acoustic signal conversion system 40, the mixing metadata separation means 61 separates the mixing metadata, and the above-described gain adjustment amount 52, speech sound / sound effect identification. Metadata 51 and multi-channel audio signal content 31 are acquired.

  Further, the channel separation means 62 receives the multi-channel sound signal content 31, separates it into a speech sound downmix signal 32 and a sound effect downmix signal 33 using the speech sound / sound effect identification metadata 51, and outputs it. .

  Here, the gain adjusting unit 63 receives the utterance sound downmix signal 32 and adjusts the signal level based on the gain adjustment amount 52. In the present embodiment, not only the signal level but also the signal levels of the speech sound downmix signal 32 and the sound effect downmix signal 33 may be adjusted as described above. Further, both the speech level and the sound effect level may be adjusted.

  Further, the gain adjusting means 63, when the input gain adjustment amount 52 includes a plurality of adjustment values, allows the user to set any adjustment value so that the hearing impaired person, the elderly, You can adjust the gain to the sound suitable for you.

  The synthesizing unit 64 synthesizes the utterance sound downmix signal 32 obtained from the gain adjusting unit 63 and the sound effect downmix signal 33 and outputs a two-channel sound signal 71.

  As described above, in the second embodiment, first, on the production side, in parallel with the production of the audio content having a large number of audio channels such as a three-dimensional audio system, the “speech sound / sound effect balance” is deteriorated. Automatic mixing metadata necessary for improvement based on attribute “speech sound / sound effect identification” data related to the semantic content of the sound signal, mixing level information of each sound channel, and measurement result of speech sound / sound effect balance Then, the program content having a number of sound channels and the accompanying mixing metadata are transmitted. On the other hand, on the receiving side, an audio signal having an optimal balance of utterance sound / effect sound can be heard by down-mixing into a 2-channel audio signal based on the audio signal of the program content and the accompanying mixing metadata. it can.

  In other words, the receiving side receives a multi-channel audio signal and accompanying mixing metadata, and downmixes it to a two-channel audio signal based on the mixing metadata, but the two-channel audio down using this mixing metadata. In the mix, “speech sound / sound effect balance” in accordance with the intention of the program producer is realized, and deterioration of “speech sound / sound effect balance” is improved.

<Additional example of “speech sound / sound effect identification”>
Next, an additional example of “speech sound / sound effect identification” as the attribute data will be described in detail.

  In the present embodiment, each acoustic channel of the transmitted multi-channel acoustic signal may include only the content of “speech sound” or “sound effect”, or may include both “speech sound” and “sound effect”. is there. Here, when only the content of “speech sound” or “sound effect” is included over the entire frequency band, the “speech sound” indicating whether the sound content is “speech sound” or “sound effect” only once in the channel. / Sound effect identification "is added to the sound signal and sent.

  On the other hand, when both “speech sound” and “sound effect” are included, for example, the frequency band is divided into a plurality of frequency bands having a preset auditory critical bandwidth, and each divided frequency band is made to correspond. Then, “speech sound / sound effect identification” indicating either “speech sound” or “sound effect” is added to the acoustic signal and sent. Therefore, in “speech sound / sound effect identification”, the frequency band signal with “speech sound” identification is regarded as a speech sound component, and the frequency band signal with “sound effect” identification is regarded as a sound effect component. Thereby, the sound signal of the channel can be separated into either the speech sound signal or the sound effect signal. It should be noted that which frequency band is set as “speech sound” and which frequency band is set as “sound effect” is set in advance according to the intention of the program producer, the contents of the sound signal, and the like.

  The above-mentioned critical bandwidth of hearing is, for example, a critical frequency table described in documents such as “E. Vicker / original author, Yukiko Yamada / translator, psychoacoustics, original title: PSYCHOKUSTIK, P. 74”, etc. Can be used.

  FIG. 7 is a diagram illustrating an example of the relationship between the critical band number and the frequency. In addition to the relationship between the critical band number and the frequency, FIG. 7 also shows the relationship between the critical bandwidth Δfg and the stop frequency fm. The critical band number z belonging to the center frequency fm is also listed. The boundary frequencies fu and f0 of the critical bands adjacent to each other belonging to the critical bandwidth correspond to the values shown in the second column.

  As shown in FIG. 7, the relationship between the band number z and the frequency f is very important for understanding the function of hearing. Therefore, using this critical bandwidth, it is possible to separate the acoustic signal of the channel into either the speech sound signal or the sound effect signal with high accuracy.

  Here, the first and second embodiments described above are not limited to this in the present invention, and may be, for example, an embodiment in which the first and second embodiments are combined.

<Sound signal conversion program>
In the above-described embodiment, the above-described acoustic signal conversion procedure according to the present invention can be performed by the dedicated device configuration in the above-described acoustic signal conversion system. For example, an acoustic signal conversion program according to the present invention can be realized by generating an execution program (acoustic signal conversion program) that can be executed and installing the acoustic signal conversion program in, for example, a general-purpose personal computer or workstation.

  That is, the acoustic signal conversion system 10 and the acoustic signal conversion devices 14 and 44 described above are a volatile storage medium such as a CPU and a RAM (Random Access Memory), a nonvolatile storage medium such as a ROM, a mouse, a keyboard, and a pointing device. The computer can include an input device such as a display unit for displaying images and data, and an interface for communicating with the outside.

  Accordingly, the functions of the acoustic signal conversion system and the acoustic signal converters 14 and 44 can be realized by causing the CPU to execute a program describing these functions. These programs can also be stored and distributed in a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like.

  That is, an execution program (acoustic signal conversion program) for causing the computer to execute the processing in each of the above-described configurations is generated, and the computer is installed on the general-purpose personal computer, server, or the like, for example. It can function as an acoustic signal conversion system or an acoustic signal conversion device.

  Here, the flow of acoustic signal conversion processing by the acoustic signal conversion program corresponding to the first and second embodiments described above will be described with reference to a flowchart or a sequence diagram.

<Sound Signal Conversion Processing Procedure: First Embodiment>
FIG. 8 is a flowchart illustrating an example of an acoustic signal conversion processing procedure according to the first embodiment. In FIG. 8, first, one or a plurality of sound signals (with attribute data such as “speech sound / sound effect identification”) are acquired (S01), and multi-channel sound signal content is produced from the obtained sound signals (S02). ). At this time, the multi-channel sound signal content is adjusted to an optimal sound by a sound adjuster or the like.

  Next, in parallel with the production of the multi-channel audio signal content, the utterance sound downmix signal and the sound effect downmix signal are generated by the conversion formula as described above (S03), and the generated utterance sound downmix signal and the effect are generated. Balance measurement is performed based on the sound downmix signal (S04). Also, a gain adjustment amount corresponding to the downmixed sound signal such as a two-channel sound is calculated from the balance measurement result obtained in the process of S04 (S05), and gain adjustment is performed based on the calculated gain adjustment amount. (S06).

  Next, the speech downmix signal and the sound effect downmix signal whose gains are adjusted by the process of S06 are synthesized (S07), and the synthesized two-channel audio signal content is output (S08).

<Sound Signal Conversion Processing Procedure: Second Embodiment>
FIG. 9 is a sequence diagram illustrating an example of an acoustic signal conversion processing procedure according to the second embodiment. In FIG. 9, first, the transmission-side acoustic signal production device 44 acquires one or a plurality of acoustic signals (with attribute data such as “speech sound / sound effect identification”) (S11), and from the obtained acoustic signals and the like. Multi-channel audio signal content is produced (S12). At this time, the multi-channel sound signal content is adjusted to an optimal sound by a sound adjuster or the like. Also, identification metadata of utterance sound / effect sound is generated (S13).

  Next, in parallel with the production of the multi-channel audio signal content, as described above, the utterance sound downmix signal and the sound effect downmix signal are generated by the conversion formula or the like (S14), and the generated utterance sound downmix signal and the effect are generated. Based on the sound downmix signal, balance measurement is performed (S15). Further, a gain adjustment amount corresponding to the down-mixed sound signal such as 2-channel sound is calculated from the balance measurement result obtained by the process of S15 (S16), and the calculated gain adjustment amount and the multi-channel sound signal content are calculated. Then, the utterance sound / sound effect identification metadata is multiplexed (S17), and the multiplexed mixing signal is transmitted to the receiving side (S18).

  Next, the receiving side acoustic signal converter 60 performs mixing metadata separation processing on the mixing signal transmitted from the acoustic signal production device 44 (S19), gain adjustment amount, multi-channel acoustic signal content, Separate speech / sound effect identification metadata. Next, channel separation processing is performed based on the separated multi-channel audio signal content and the speech / sound effect identification metadata (S20), and the speech sound downmix signal and the sound effect downmix signal are obtained and obtained. The uttered sound downmix signal is further adjusted by the gain adjustment amount obtained in S19 (S21). Next, the speech downmix signal and the sound effect downmix signal whose gains are adjusted by the processing of S21 are synthesized (S22), and the synthesized two-channel sound signal content is output (S23).

  As described above, the acoustic signal conversion process described above can be easily realized by installing the execution program in the computer.

  As described above, according to the present invention, when a multi-channel sound signal is downmixed, the sound can be converted into an optimum sound without deteriorating the balance between the uttered sound and the sound effect. Specifically, when the conventional downmix is performed, the “speech sound / sound effect balance” deteriorates, for example, it becomes difficult to hear the narration sound due to the BGM sound. In some cases, the “balance of sound effects” may not have been achieved. However, by applying the present invention, an appropriate “speech sound / It is possible to automatically produce a downmix of a 2-channel sound signal, 5.1-channel surround sound signal, etc. in which the “balance of sound effects” is maintained.

  In the sound signal conversion apparatus, by applying the present invention, the sound signal of the program content having a large number of sound channels such as a three-dimensional sound system and the mixing metadata associated therewith are received, and the two-channel sound is received. When downmixing to a signal, 5.1 channel surround sound signal, or the like, it is possible to improve the deterioration of the “speech sound / sound effect balance” generated by the specified downmix.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

10, 40 Acoustic signal conversion system 11 Sound recording / reproducing device 12 Microphone (voice input means)
DESCRIPTION OF SYMBOLS 13,43 Acoustic mixing apparatus 14,60 Acoustic signal converter 21 Speech / effect sound balance measurement apparatus 22 Gain adjustment amount calculation means 23,63 Gain adjustment means 24,64 Synthesis means 31 Speech sound downmix signal 32 Effect sound downmix Signal 33 Multi-channel audio signal content 34 2-channel audio signal content 44 Audio signal production device 45 Mixing metadata multiplexing means 51 Speech / effect sound identification metadata 52 Gain adjustment amount 61 Mixing metadata separating means 62 Channel separating means

Claims (6)

In an acoustic signal conversion device that converts an acoustic signal composed of a first signal level and a second signal level in correspondence with a preset number of channels,
The acoustic signal having the first signal level when the acoustic signal corresponding to the first channel number is downmixed to the acoustic signal corresponding to the second channel number, and the acoustic signal having the second signal level. A balance measuring means for measuring the signal level mix balance;
A gain adjustment amount of the first signal level or the second signal level is calculated in correspondence with a relative level difference between the first signal level and the second signal level obtained by the balance measuring means. Gain adjustment amount calculating means;
Gain adjusting means for adjusting the gain of the first signal level and / or the second signal level based on the gain adjustment amount obtained by the gain adjustment amount calculating means;
A sound signal corresponding to the second number of channels is obtained by synthesizing the sound signal having the first signal level and the sound signal having the second signal level using the sound signal having the gain adjusted obtained by the gain adjusting means. It has a synthesizing means for outputting a signal,
The gain adjustment means sets an adjustment time having a length corresponding to the amount of increase or decrease of the gain adjustment amount obtained by the gain adjustment amount calculation means, and is set by the combining means at the set adjustment time. An acoustic signal converter characterized by adjusting a gain so that a total volume after synthesis is kept constant . In the acoustic signal converter for converting the acoustic signal composed of the first signal level and the second signal level transmitted from the production side in correspondence with the preset number of channels,
From the multiplexed signal transmitted from the production side, an acoustic signal corresponding to the first channel number, the first signal level when downmixed from the first channel number to the second channel number, and To identify the first signal level or the gain amount of the second signal level corresponding to the mix balance of the second signal level, and the acoustic signal of the first signal level and the second signal level Mixing metadata separation means for separating the identification metadata into,
The acoustic signal corresponding to the first number of channels obtained by the mixing metadata separation unit is down-mixed to the acoustic signal corresponding to the second number of channels using the identification metadata. Channel separation means for separating an acoustic signal having a signal level of 1 and an acoustic signal having the second signal level;
A gain for adjusting a gain of the first signal level and / or the second signal level based on a gain adjustment amount corresponding to a relative level difference between the first signal level and the second signal level. Adjusting means;
A sound signal corresponding to the second number of channels is obtained by synthesizing the sound signal having the first signal level and the sound signal having the second signal level using the sound signal having the gain adjusted obtained by the gain adjusting means. Combining means for outputting a signal,
The gain adjusting means sets an adjustment time having a length corresponding to the amount of increase or decrease of the gain adjustment amount, and the total volume after synthesis by the synthesizing means is kept constant for the set adjustment time. An acoustic signal converter characterized by adjusting a gain so as to sag . The synthesis means includes
The acoustic signal conversion according to claim 1 or 2 , wherein a volume of the acoustic signal corresponding to the second number of channels is adjusted according to a change amount of the acoustic signal gain-adjusted by the gain adjusting unit. apparatus. The acoustic signal conversion apparatus according to any one of claims 1 to 3 , wherein the acoustic signal composed of the first signal level and the second signal level is a speech sound and a sound effect. An acoustic signal composed of the input first signal level and the second signal level is separated into the uttered sound or the sound effect using a preset critical bandwidth of hearing. The acoustic signal converter according to claim 4 . An acoustic signal conversion program for causing a computer to function as each unit included in the acoustic signal converter according to any one of claims 1 to 5 .