KR102049603B1 - Apparatus and method for providing the audio metadata, apparatus and method for providing the audio data, apparatus and method for playing the audio data - Google Patents

Abstract

An audio metadata generator for generating audio metadata including channel information of audio raw data; And an audio metadata transmitter for transmitting the generated audio metadata to an audio data reproducing apparatus.

Description

Apparatus and method for providing audio metadata, Apparatus and method for providing audio data, Apparatus and method for reproducing audio data {APPARATUS AND METHOD FOR PROVIDING THE AUDIO METADATA, APPARATUS AND METHOD FOR PROVIDING THE AUDIO DATA, APPARATUS AND METHOD FOR PLAYING THE AUDIO DATA}

The present invention relates to a method for generating audio data having multichannels and providing the same to the audio data reproducing apparatus. More particularly, the present invention relates to a method for representing multichannel audio data using metadata including channel information of an audio signal. It is about a method.

In order to implement audio data in multiple channels, information on how many signals the audio content is made of, and which channel should be placed in a space, is required. The existing 5.1-channel audio data is based on the condition that a total of six signals are produced and reproduced at 0, +30, +110, +250, +330, and null positions.

UHDTV technology has been developed, and research on more realistic audio playback methods using more speakers than the 5.1 channels provided by HDTV has received much attention. In addition, there are actually various playback methods of 5.1 channels or more.

In relation to a conventional method of representing audio data having a multi-channel, Korean Patent No. 10-0522593 generates a matched output with a conventional multi-channel audio when a plurality of outputs are added in a multi-channel audio system. In this paper, a multi-channel audio realization method that implements natural stereo sound is proposed.

Specifically, the multi-channel audio implementation method includes receiving and decoding an encoded audio stream, and generating multi-channel stereophonic sound using the decoded audio stream. Next, the multi-channel audio implementation method includes generating a television left speaker output and a television right speaker output using a left stereo channel signal, a right stereo channel signal, and a center channel signal in the generated multichannel sound output. Provide multichannel audio with additional channels.

However, the conventional multichannel audio implementation method describes only a configuration for converting a 5.1 channel to a 7.1 channel, and there is a disadvantage in that it is difficult to configure an audio channel system of various methods.

In addition, various methods such as Auro 3D and T.Holman 12.2 (10.2) channel present six or more speakers in a position other than the horizontal plane to express audio channels in a unique manner, but a separate additional system such as an A / V receiver is provided. It has the disadvantage of being necessary.

Therefore, there is a need for a technology capable of representing audio data having multichannels in various ways.

The present invention provides an audio metadata providing apparatus capable of generating audio metadata including channel information of audio raw data and transmitting the same to audio reproduction apparatus.

The present invention provides an audio data providing apparatus that combines audio raw data and audio metadata including channel information of audio raw data and transmits the same to the audio reproducing apparatus to output multichannel audio in various ways.

The present invention provides an audio data reproducing apparatus capable of demultiplexing or decoding audio data received from an audio data providing apparatus to reproduce audio raw data according to audio metadata.

According to the present invention, when a channel included in the reproduction environment information is a lower channel than a channel included in the audio row data, the present invention provides an apparatus for reproducing an audio data compatible with a channel by combining a signal of an audio channel included in the audio row data. do.

An apparatus for providing audio metadata according to an embodiment of the present invention includes an audio metadata generator for generating audio metadata including channel information of audio raw data; And an audio metadata transmitter for transmitting the generated audio metadata to the audio data reproducing apparatus.

An audio data providing apparatus according to an embodiment of the present invention comprises: an audio metadata generator for generating audio metadata including channel information of audio row data; An audio data combiner for combining the audio row data and the generated audio metadata into one audio data; And an audio data transmitter for transmitting the combined audio data to the audio data reproducing apparatus.

An audio data reproducing apparatus according to an embodiment of the present invention comprises: an audio data receiving unit for receiving audio data from an audio data providing apparatus; An audio data analyzer configured to analyze audio row data and audio metadata from the received audio data; And an audio data reproducing unit for reproducing the audio raw data based on the analyzed audio metadata and preset reproduction environment information.

According to an embodiment of the present invention, audio metadata including channel information of audio raw data may be generated and transmitted to the audio reproducing apparatus.

According to an embodiment of the present invention, audio metadata including audio row data and channel information of audio row data may be combined and transmitted to the audio reproducing apparatus to output multichannel audio in various ways.

According to an embodiment of the present invention, the audio raw data may be reproduced according to the audio metadata by demultiplexing or decoding the audio data received from the audio data providing apparatus.

According to an embodiment of the present invention, when a channel included in the reproduction environment information is a lower channel than a channel included in the audio row data, channel compatibility is possible through a combination of signals of the audio channel included in the audio row data. Do.

1 is a view for explaining the configuration of audio data according to an embodiment of the present invention.
2 is a diagram illustrating an operation of transmitting audio metadata to an audio data reproducing apparatus by an audio metadata providing apparatus according to an embodiment of the present invention.
3 is a diagram illustrating an operation of transmitting audio data to an audio data reproducing apparatus by an audio data providing apparatus according to an embodiment of the present invention.
4 is a diagram illustrating a configuration of audio metadata according to an embodiment of the present invention.
5 is a view for explaining an embodiment of the audio metadata configuration according to an embodiment of the present invention.
6 is a diagram for describing a criterion of audio channel position information according to an embodiment of the present invention.
7 is a diagram in which a speaker is placed in a space based on audio channel position information according to an embodiment of the present invention.
8 is a view for explaining another embodiment of the audio metadata configuration of the present invention.
9 is a flowchart illustrating an operation of providing audio metadata by an audio metadata providing apparatus according to an embodiment of the present invention.
10 is a flowchart illustrating an operation of providing audio data by an audio data providing apparatus according to an embodiment of the present invention.
11 is a flowchart illustrating an operation of reproducing audio data by an audio data reproducing apparatus according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The audio metadata providing method according to an embodiment of the present invention may be performed by an audio metadata providing apparatus, and the audio data providing method according to an embodiment of the present invention may be performed by an audio data providing apparatus. In addition, the audio data reproduction method according to an embodiment of the present invention may be performed by the audio data reproduction apparatus. Like reference numerals in the drawings denote like elements.

1 is a view for explaining the configuration of audio data according to an embodiment of the present invention.

Referring to FIG. 1, the audio data 110 may include audio row data 120 and audio metadata 130. The audio raw data refers to an audio signal and may include as many audio signals as the number of channels. The audio raw data 120 may sequentially store the produced audio signal. For example, in the case of audio data 110 of 5.1 channels, audio low data 120 includes L (Left), R (Right), C (Center), LFE (Low Frequency Effect), LS (Left Side), Each signal 140 associated with a right side (RS) channel may be included.

The audio metadata 130 refers to representation information related to the audio signal, and may include channel information of the audio raw data 120. Detailed description thereof will be described later with reference to FIG. 4.

2 is a view for explaining an operation of transmitting audio metadata to an audio data reproducing apparatus by an audio metadata providing apparatus according to an embodiment of the present invention.

Referring to FIG. 2, the audio metadata providing apparatus 210 may include an audio metadata generator 220 and an audio metadata transmitter 230.

The audio metadata generator 220 may generate audio metadata including channel information of audio row data. The generated audio metadata may be managed separately from the audio row data and may be coded by the audio metadata providing device 210.

The audio metadata transmitter 230 may transmit audio metadata to the audio data reproducing apparatus 250. According to another embodiment, the audio metadata transmitter 230 may multiplex audio row data and audio metadata received from the audio row data providing apparatus 240 and transmit the multiplexed audio row data and the audio metadata to the audio data reproducing apparatus 250. Here, multiplexing refers to a method of combining multiple signals into a single signal.

The audio raw data providing apparatus 240 may transmit audio raw data in which audio signals are collected for each channel to the audio data reproducing apparatus 250. In some cases, the audio metadata providing apparatus 210 may generate audio metadata based on the audio raw data in cooperation with the audio raw data providing apparatus 240.

The audio data reproducing apparatus 250 may reproduce the audio raw data by analyzing the received audio raw data and the audio metadata. At this time, the audio data reproducing apparatus 250 may reproduce the audio row data according to the channel information based on the audio environment setting information and the audio metadata stored in the audio data reproducing apparatus 250.

3 is a diagram for describing an operation of transmitting audio data to an audio data reproducing apparatus by an audio data providing apparatus according to an embodiment of the present invention.

Referring to FIG. 3, the audio data providing apparatus 310 may include an audio metadata generator 320, an audio data combiner 330, and an audio data transmitter 340.

The audio metadata generator 320 may generate audio metadata including channel information of audio row data. That is, the audio metadata generator 320 may analyze the audio row data to represent channel information of the audio signal as audio metadata.

The audio data combiner 330 may combine the audio raw data and the audio metadata into one audio data. In this case, the audio data combiner 330 may code only audio row data or audio metadata, or code audio row data and audio metadata together.

The audio data transmitter 340 may transmit the combined audio data to the audio data reproducing apparatus 350. In other words, the audio data transmitter 340 may multiplex the coded audio data and transmit the multiplexed data to the audio data reproducing apparatus 350.

The audio data reproducing apparatus 350 may include an audio data receiving unit 360, an audio data analyzing unit 370, and an audio data reproducing unit 380.

The audio data receiver 360 may receive audio data from the audio data providing apparatus 310.

The audio data analyzer 370 may analyze audio row data and audio metadata from the received audio data. The audio data analyzer 370 may demultiplex or decode the audio data. The audio data analyzer 370 may separate audio row data and audio metadata from the audio data through demultiplexing and decoding processing, and may extract an audio signal for each channel from the audio row data.

According to another embodiment of the present invention, the audio data analyzer 370 separates the received audio data into audio raw data and audio metadata, and the user adds an additional audio signal using the separated audio raw data and the audio metadata. You can edit and reorganize.

The audio data reproducing unit 380 may reproduce the audio raw data based on the analyzed audio metadata and preset reproduction environment information. Here, the preset playback environment refers to audio environment setting information of the audio reproducing apparatus such as a speaker arrangement situation of a home TV system or an A / V receiver.

If the channel included in the playback environment information is a lower channel than the channel included in the audio raw data, the audio data reproducing unit 380 includes the audio included in the audio raw data according to the backward compatibility information of the audio channel included in the audio metadata. The signal of the channel can be combined to change to the lower channel.

For example, when the number of channels included in the audio row data is 6, and the number of set channels included in the reproduction environment information of the audio reproducing apparatus is 2, the audio data reproducing unit 380 converts the received audio data into two channels. Down mixing can be done with the lower channel. Here, downmixing means converting audio data in which several channels are combined into audio data having a low number of channels. Through this, the audio data reproducing apparatus 350 may reproduce the audio data through channel compatibility even if the audio data having more channels than the channels included in the audio configuration information is received.

When the audio data reproducing apparatus 350 receives audio data having a lower channel than the channel included in the audio configuration information, the audio data reproducing apparatus 350 does not output the channel that does not include the audio signal, thereby reproducing the audio data regardless of the number of channels. Can play.

4 is a diagram illustrating a configuration of audio metadata according to an embodiment of the present invention.

Referring to FIG. 4, the audio metadata 410 may include audio channel number information 420, audio channel name information 430, audio channel location information 440, or audio channel backward compatibility information 450. It may include at least one of.

The number information 420 of audio channels indicates the number of audio signals included in audio row data. For example, if the 5.1 channel signal is included in the audio row data, the number information 420 of the audio channel has a value of 6. Also, if the 7.1 channel signal is included in the audio row data, the number information 420 of the audio channel has a value of 8. In some cases, the number information 420 of the audio channel may indicate the number of objects of the audio signal stored on an object basis.

The name information 430 of the audio channel indicates a channel name of each audio signal included in the audio row data. For example, in the case of the 5.1-channel signal, the name information 430 of the audio channel may include information (L, R, C, LFE, LS, RS). This means that the audio signals stored in the audio row data correspond to the names of L, R, C, LFE, LS, and RS in order. Here, the definition of the audio channel name and the order of the audio signals is not mandatory, but can be arbitrarily set by the producer. However, for compatibility with the existing system, it is preferable to use the 5.1 channel and 2.0 channel signals unified in the form of (L, R, C, LFE, LS, RS) and (L, R), respectively.

The location information 440 of the audio channel indicates in which position the audio signal channel should be spatially located. The position information 440 of the audio channel may be composed of horizontal azimuth information and vertical azimuth information.

As shown in FIG. 6, when the user 610 faces the front side, the horizontal azimuth angle is set to 0 degrees 620, which is a reference point with respect to the front surface with respect to the surface horizontal to the ground, and the reference is set with the clockwise direction as the (+) direction. Can be set. Vertical Azimuth Angle As shown in FIG. 6, when the user 630 faces the front face, the reference point is set to 0 degree 640 which is a reference point with respect to the front face perpendicular to the ground, and the upper direction is set as (+) direction. Can be set.

For example, in the case of 2.0 channels, if the audio channel name information 430 is represented as (L, R) and the audio channel position information 440 is represented as [(330, 0), (30, 0)] Indicates that the L channel is disposed at 330 degrees horizontally and 0 degrees vertically, and the R channel is disposed at 30 degrees horizontally and 0 degrees vertically. If it does not matter in any position, it can be displayed using a null symbol and can be represented as (null, null) as an azimuth.

The backward compatibility information 450 of the audio channel represents a method of changing the lower channel by combining the signals of the audio channel included in the audio row data. For example, the backward compatibility information 450 may indicate how audio row data having 7.1 channels may be compatible with 5.1 or 2.0 channels. The audio data having a larger number of channels than the 5.1 channel may include information on changing to 5.1 or 2.0 channels, and the audio data of 5.1 channels need only include information on compatibility with 2.0 channels.

5 is a view for explaining an embodiment of the audio metadata configuration according to an embodiment of the present invention.

According to FIG. 5, the audio metadata 510 includes number information 520 of audio channels, name information 530 of audio channels, location information 540 of audio channels, and backward compatibility information 550 of audio channels. Doing.

The value of the channel number information 520 of the audio is 6, and it can be seen that the number of channels of the audio raw data is 6 and the LFE channel is 1, 5.1 channels. In addition, it can be seen that the channel names of the audio signals included in the audio row data are L, R, C, LFE, LS, and RS in order according to the name information 530 of the audio channel.

In the audio channel position information 540, the L channel becomes (330, 0) and is disposed at 330 degrees horizontally and vertically 0 degrees, and the R channel becomes (30, 0) and is disposed at 30 degrees horizontally and 0 degrees vertically. . The C channel is arranged at 0 degrees horizontal and 0 degrees vertically at (0, 0), and the LS channel is arranged at 250 degrees horizontal and 0 degrees vertically at (250, 0). In addition, the RS channel becomes (110, 0), and is disposed at 110 degrees horizontally and 0 degrees vertically, and the LFE channel is (null, null) and may be disposed at any position. Audio channel position information of 5.1-channel and 2.0-channel signals is [(330, 0), (30, 0), (0, 0), (null, null), (250, 0), respectively, for compatibility with existing systems. , (110, 0)], [(330, 0), (30, 0)].

The implementation of this in the real space is represented in FIG. FIG. 7 illustrates a layout of a speaker based on the audio channel position information 540 of FIG. 5.

Referring to FIG. 7, a speaker is positioned at a position where each channel should be arranged around the user 710. The multi-channel audio data is expressed based on the information included in the audio metadata. By using this, multi-channel audio data having more than 5.1 channels can be implemented in various ways.

In FIG. 5, the backward compatibility information 550 of the audio channel shows a constant 560 and a calculation equation 570 for downmixing from 5.1 channel to 2.0 channel. The constant (a, k) 560 is an arbitrary constant made by the audio data producer, and the calculation equation 570 may be stored according to the sub-channel order. In Equation 570, D (1) means a, which is the first data of (D), and D (2) means k, which is the second data of (D). N (1) means the L channel signal as the first data of (N), and N (2) means the R channel signal as the second data of (N). N (3) denotes a C channel signal, N (4) denotes an LFE channel signal, N (5) denotes an LS channel signal, and N (6) denotes an RS channel signal. That is, the backward compatibility information 550 of the audio channel may store downmixing information as shown in Table 1 below. The above embodiment merely describes an example of downmixing from 5.1 channel to 2.0 channel, and various modifications and arbitrary settings are possible.

Channel layout Channel matrixing
(a and k are arbitrary constants) 5.1-> 2.0

8 is a view for explaining another embodiment of the audio metadata configuration of the present invention.

According to FIG. 8, the audio metadata 810 is as the number information 820 of the audio channel, the name information 830 of the audio channel, the location information 840 of the audio channel, and the backward compatibility information 850 of the audio channel. Contains compatibility information for all three channels.

The value of the channel number information 820 of audio is 12, and it can be seen that the number of channels of audio raw data is 12 and that there are 10.2 channels having two LFE channels. Also, according to the name information 830 of the audio channel, each channel name of the audio signals included in the audio row data is sequentially L, R, C, LH, RH, LS, RS, LB, RB, TC, LFE1, LFE2. It can be seen that.

Looking at the audio channel position information 840, the L channel becomes (330, 0) and is disposed at 330 degrees horizontally and 0 degrees vertically, and the R channel is disposed at 30 degrees horizontally and 0 vertically at (30, 0), The C channel is (0, 0) at 0 degrees horizontal and 0 degrees vertical. Subsequently, the LH channel becomes (330, 30) and is disposed at 330 degrees horizontal and 30 degrees vertically, and the RH channel is arranged at 30 degrees horizontal and 30 degrees vertically at (30, 30), and the LS channel is (270, 30). 0) horizontally and 270 degrees vertically 0 degrees. In addition, the RS channel becomes (90, 0) and is disposed at 90 degrees horizontal and vertical 0 degrees, and the LB channel is arranged at 210 degrees horizontal and 0 degrees vertically at (210, 0) and the RB channel is (150, 0). ) Is placed at 150 degrees horizontally and 0 degrees vertically. The TC channel is (0, 90) horizontally at 0 degrees and vertical 90 degrees, and the LFE1 channel and the LEF2 channel are (null, null), indicating that they may be disposed at any position. As such, the audio metadata 810 may include not only the horizontal plane but also audio channel information on a layer higher than the horizontal plane, such that the audio producer may configure a three-dimensional sound system.

The backward compatibility information 850 of the audio channel shows a constant 860 and a calculation equation 870 for downmixing from 10.2 channels to 7.1 channels, 5.1 channels, and 2.0 channels. The constants 860 (a1, c1), (a2, c2) (a3, c3) are arbitrary constants created by the audio data producer, and the downmixing formula information 870 may be stored according to the order of the lower channels. Each downmixing formula information may include signal combination and conversion information for changing from 10.2 channels to each channel. Thus, even if the audio reproducing apparatus receives the 10.2 channel audio data, the audio reproducing apparatus can reproduce the audio data compatible with the channel configuration of the audio reproducing apparatus. The order in which the downmixing formula information is stored may be arbitrarily set by the producer and is not limited to the above embodiment.

9 is a flowchart illustrating an operation of providing audio metadata by an audio metadata providing apparatus according to an embodiment of the present invention.

In operation 910, the audio metadata providing apparatus may generate audio metadata including information related to audio row data. The audio metadata generated by the audio metadata providing apparatus may be managed separately from the audio row data, and may be coded by the audio metadata providing apparatus.

In operation 920, the audio metadata providing apparatus may transmit the generated audio metadata to the audio data reproducing apparatus. According to another exemplary embodiment, the audio metadata providing apparatus may multiplex audio row data and audio metadata received from the audio row data providing apparatus and transmit the multiplexed audio row data to the audio data reproducing apparatus.

10 is a flowchart illustrating an operation of providing audio data by an audio data providing apparatus according to an embodiment of the present invention.

In operation 1010, the audio data providing apparatus may generate audio metadata including information related to audio row data. That is, the audio data providing apparatus may represent information related to the audio signal as audio metadata based on the audio raw data.

In operation 1020, the audio data providing apparatus may combine the audio raw data and the generated audio metadata into one audio data. In this case, the audio data providing apparatus may separately code the audio row data and the audio metadata or may code the audio row data and the audio metadata together.

In operation 1030, the audio data providing apparatus may transmit the combined audio data to the audio data reproducing apparatus. The audio data providing apparatus may multiplex the coded audio data and transmit the multiplexed audio data to the audio data reproducing apparatus.

11 is a flowchart illustrating an operation of reproducing audio data by an audio data reproducing apparatus according to an embodiment of the present invention.

In operation 1110, the audio data reproducing apparatus may receive audio data from the audio data providing apparatus.

In operation 1120, the audio data reproducing apparatus may analyze the audio raw data and the audio metadata from the received audio data. The audio data reproducing apparatus may demultiplex or decode the audio data. The audio reproducing apparatus may separate audio row data and audio metadata from the audio data through demultiplexing and decoding processing, and extract an audio signal for each channel from the audio row data.

According to another embodiment of the present invention, the audio data reproducing apparatus divides the received audio data into audio raw data and audio metadata, and the user edits an additional audio signal using the separated audio raw data and audio metadata. I can reconstruct

In operation 1130, when the channel included in the reproduction environment information is a lower channel than the channel included in the audio raw data, the audio data reproducing apparatus may determine the audio raw data according to the backward compatibility information of the audio channel included in the audio metadata. The signal of the included audio channel may be combined to change to a lower channel.

In operation 1140, when the channel included in the reproduction environment information is the same channel or higher than the channel included in the audio row data, the audio data reproducing apparatus is based on the analyzed audio metadata and the preset reproduction environment information. Audio raw data can be played back. In addition, the audio data reproducing apparatus may reproduce the audio row data changed to the lower channel in step 1130.

Methods according to an embodiment of the present invention can be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. A computer readable medium is any data storage device that can store data that can later be read by a computer system. Examples of computer readable media include read only memory, random access memory, CD-ROM, DVD, magnetic tape, optical data storage, and carrier waves. The computer readable medium may also work distributed over network-connected computer systems such that the computer readable code is stored and executed in a distributed fashion. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims below, but also by those equivalent to the claims.

110: audio data
120: audio raw data
130: audio metadata
140: audio signal

Claims (10)

Generating audio metadata including channel information of the audio data;
Providing the audio data and the generated audio metadata
Including,
The audio data is data associated with a channel,
The audio metadata,
And at least one of audio channel number information, audio channel name information, and audio channel position information. The method of claim 1,
The audio metadata,
The audio data providing method further comprising backward compatibility information of the audio channel. Receiving audio data;
Playing the audio data based on the audio metadata of the audio data
Including,
The audio metadata,
And at least one of audio channel number information, audio channel name information, and audio channel position information. The method of claim 3,
The audio metadata,
Audio data playback method further comprising the backward compatibility information of the audio channel. The method of claim 3,
The number information of the audio channel,
A method of playing audio data, the number of audio channels associated with the audio data. The method of claim 3,
Position information of the audio channel,
A method of reproducing audio data, which is information indicating where audio channels should be spatially located. The method of claim 3,
Position information of the audio channel,
An audio data reproduction method comprising horizontal azimuth information and vertical azimuth information. The method of claim 3,
Name information of the audio channel,
A method of playing audio data, each of which is a name of audio channels included in the audio data. The method of claim 4, wherein
The backward compatibility information of the audio channel is
And a method of changing the signal of the audio channel included in the audio data into a lower channel. Identifying audio metadata of the audio data;
Extracting detailed information included in the audio metadata
Including,
The audio metadata,
And at least one of audio channel number information, audio channel name information, and audio channel position information.

Images