KR101005303B1 - Automatic output pattern generation method and device harmonized with the characteristics of audio data - Google Patents

Abstract

PURPOSE: An automatic output pattern generation method harmonized with the characteristics of audio data, and a device thereof are provided to enable the visual display for currently-reproduced audio data regardless of the kind of audio data. CONSTITUTION: An automatic output pattern generation device harmonized with the characteristics of audio data comprises an audio frame division unit(410), a frame level calculation unit(420) and an output pattern control unit(430). The audio frame division unit receives the audio data of which sound source is reproduced and divides the audio data into at least one audio frame. The frame level calculation unit calculates the frame level for each audio frame, and the output pattern control unit generates an output pattern corresponding to the output pattern setting information set according to a parameter set in correspondence to the frame level.

Description

Automatic output pattern generation method and device harmonized with the characteristics of audio data}

The present invention relates to a method and apparatus for automatically generating an output pattern to respond according to the nature of audio data.

Audio reproducing apparatuses for reproducing audio data, such as audio files stored in advance and audio streaming data transmitted through a network such as a wired or wireless Internet, are widely used. The audio data is decoded into PCM (Pulse Code Modulation) data and the like, and the audio reproducing apparatus converts the digital audio data into an analog audio signal and outputs it through an audio output unit such as a speaker or earphone. Here, the audio reproducing apparatus is a concept including not only an audio reproducing apparatus such as a CD player, an MP3 player, etc., but also a general-purpose apparatus having an audio data reproducing function such as a general-purpose computer, a laptop computer, a mobile communication terminal, a smart phone, and the like.

With the recent development of display technology, an audio reproducing apparatus is provided with or connected to various display units such as LCDs, LEDs, OLEDs, etc., so that information related to audio data being reproduced can be visually output to a user.

However, the conventional audio reproducing apparatus shows only the pieces of information (play time, album name, file name, artist, genre, album picture, etc.) of the corresponding audio data through the display unit when the audio data is being played or the equalizer function. Because it only performs, due to its monotony there is a limit that can be limited to induce user interest.

Visual playback refers to visualization that displays various shapes and the like through a display unit of an audio reproducing apparatus.

The visualization according to the prior art merely displays a geometric or metaphysical figure on the display unit, and has a limitation in that it does not respond to the acoustic reproduction of audio data being reproduced.

The above-described background technology is technical information that the inventor holds for the derivation of the present invention or acquired in the process of deriving the present invention, and can not necessarily be a known technology disclosed to the general public prior to the filing of the present invention.

According to the present invention, an output pattern is automatically matched with the characteristics of the audio data for setting an output pattern such as a display screen and / or vibration outputted through a pattern output unit using a frame level obtained by analyzing audio data being reproduced in a sound frame unit. It is to provide a generating method, an automatic output pattern generating device and an audio reproducing device.

In addition, the present invention can show a variety of display screens to the user by changing the characteristics of the objects in the display screen, performing the screen change or by changing the dynamic object speed and / and frame rate according to the nature of the audio data, causing the user's interest and An object of the present invention is to provide an automatic output pattern generating method, an automatic output pattern generating device, and an audio reproducing device that are consistent with the characteristics of audio data that can be kept interesting.

In addition, the present invention outputs a vibration suitable to the characteristics of the audio data for a portable terminal having a vibration means such as a mobile communication terminal or a smart phone and the nature of the audio data capable of tactile expression of the audio data being reproduced and An object of the present invention is to provide a method for automatically generating an output pattern, an apparatus for automatically generating an output pattern, and an audio reproducing apparatus.

Other objects of the present invention will be readily understood through the following description.

According to an aspect of the present invention, there is provided an apparatus for automatically generating an output pattern output through an output device corresponding to audio data being reproduced.

An apparatus for automatically generating an output pattern according to an embodiment may include: an audio frame divider configured to receive audio data being reproduced in the sound source and divide the audio data into one or more audio frames, wherein the size of the audio frame corresponds to an output capability of the output device; Has a determined size; A frame level calculator for calculating a frame level for each of the audio frames; And an output pattern controller configured to generate an output pattern corresponding to output pattern setting information set according to a parameter set corresponding to the frame level, wherein the audio data is a digital signal decodable according to a pulse code modulation (PCM) scheme. The frame level may be an average of PCM levels of one or more audio samples belonging to each of the audio frames.

The output device may be a display device, and the output pattern may be applied to a display screen on which a background is determined according to the output pattern setting information and one or more dynamic objects are disposed and displayed on the display device.

The input audio data functions as audio data for setting a display screen, and the parameter is a speed parameter for adjusting a frame rate, which is a virtual reproduction speed of the display screen setting audio data as a reference for setting the output pattern. And a speed parameter setting unit configured to set the speed parameter by adjusting a time stamp of the one or more audio frames included in the display screen setting audio data corresponding to an average of frame levels of audio frames belonging to a certain time unit in the past. It may further include.

delete

The parameter may be an object characteristic parameter for adjusting one or more of the size and rotation amount of the dynamic object, and may further include an object characteristic parameter setting unit for setting the object characteristic parameter corresponding to the frame level.

Alternatively, the parameter is a screen change parameter for a time point when the display screen is changed, and the display screen changes a time point when a predetermined time elapses from a previous screen change time point and a frame level of an audio frame to be determined becomes equal to or greater than a reference value. The apparatus may further include a screen change parameter setting unit configured to set the screen change parameter to be determined as a time point. The reference value may be a value obtained by converting a reference constant related to a frequency of a screen change point corresponding to an average of frame levels of audio frames belonging to a predetermined time unit based on the audio frame to be determined.

Alternatively, the parameter may be a speed parameter for adjusting the speed of the dynamic object, and may further include a speed parameter setting unit configured to set the speed parameter corresponding to an average of frame levels for audio frames belonging to a certain time unit in the past. The speed of the dynamic object may be a positional displacement per hour of the dynamic object.

The output device is a vibration generating device, and the output pattern may be applied to vibration generated by the vibration generating device by varying one or more of intensity and pattern according to the output pattern setting information.

The frame level calculator performs Fourier transform on the audio data to calculate the frequency band level, and the output pattern setting unit defaults or calculates the frequency band level calculated for one or more frequency bands selected by a user. The output pattern setting information for generating the vibration may be generated when the threshold value is greater than or equal to.

The output pattern setting unit may generate the output pattern setting information for applying at least one of the vibration intensity and the pattern according to the frequency band level.

On the other hand, according to another aspect of the present invention, a method for automatically generating an output pattern in the audio pattern reproduction device that is provided in the audio reproduction device and output through the output device to correspond to the characteristics of the audio data reproduced in the audio reproduction device and There is provided a recording medium having recorded thereon a program for carrying out this method.

According to an embodiment, the method of automatically generating an output pattern includes: (a) dividing audio data selected by a user into one or more audio frames, wherein the size of the audio frame is determined according to an output capability of the output device. Has; (b) calculating a frame level for each of the audio frames; And (c) setting and outputting output pattern setting information according to a parameter set according to the frame level, wherein an output pattern corresponding to the output pattern setting information is output through the output device, and the audio data is output. Is a decodable digital signal according to a pulse code modulation (PCM) scheme, and the frame level may be an average of PCM levels of one or more audio samples belonging to each of the audio frames.

The output device may be a display device, and the output pattern may be applied to a display screen on which a background is determined according to the output pattern setting information and one or more dynamic objects are disposed and displayed on the display device.

The input audio data functions as audio data for setting a display screen, and the parameter is a speed parameter for adjusting a frame rate, which is a virtual reproduction speed of the display screen setting audio data as a reference for setting the output pattern. Before the step (c), the speed is adjusted by adjusting a time stamp of the one or more audio frames included in the display screen setting audio data corresponding to an average of frame levels for audio frames belonging to a certain time unit in the past. The method may further include setting a parameter.

delete

The parameter may be an object characteristic parameter for adjusting one or more of the size and rotation amount of the dynamic object, and may further include setting the object characteristic parameter corresponding to the frame level before the step (c). .

Or the parameter is a screen change parameter relating to a point in time at which the display screen is changed, and before step (c), a predetermined time has elapsed since the previous screen change point, and the frame level of the audio frame to be determined is equal to or greater than a reference value. The method may further include setting the screen change parameter to determine a point in time at which the display screen is to be changed. The reference value may be a value obtained by converting a reference constant related to a frequency of a screen change point corresponding to an average of frame levels of audio frames belonging to a predetermined time unit based on the audio frame to be determined.

Or the parameter is a speed parameter for adjusting the speed of the dynamic object, and before the step (c), setting the speed parameter corresponding to an average of frame levels for audio frames belonging to a certain time unit in the past. It may further include. The speed of the dynamic object may be a positional displacement per hour of the dynamic object.

The output device is a vibration generating device, and the output pattern may be applied to vibration generated by the vibration generating device by varying one or more of intensity and pattern according to the output pattern setting information.

Calculating a frequency band level by performing a Fourier transform on the audio data before step (c), wherein step (c) is performed for one or more frequency bands designated by default or selected by a user. When the calculated frequency band level is greater than or equal to a predetermined threshold value, the vibration may be generated.

Prior to the step (a), the step of leveling the volume of the audio data to a predetermined level may be carried out.

Other aspects, features, and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to an embodiment of the present invention, it is possible to set an output pattern such as a display screen and / or a vibration output through the pattern output unit by using the frame level obtained by analyzing audio data being reproduced in a sound source unit in audio frame units. Regardless of the type of the audio data, there is an effect capable of visualizing the presentation of the audio data being played.

In addition, according to the characteristics of the audio data, various display screens can be shown to the user by changing object characteristics, performing screen changes, or changing the dynamic object speed and / or frame rate according to the nature of the audio data. There is a possible effect.

In addition, a portable terminal having a vibration means such as a mobile communication terminal or a smart phone also outputs a vibration suitable for the characteristics of the audio data, thereby enabling a tactile expression of the reproduced audio data.

1 is a view schematically showing the configuration of an audio reproducing apparatus according to an embodiment of the present invention.
2 is a view schematically showing the configuration of an audio reproduction unit according to an embodiment of the present invention.
3 is an exemplary view of a display screen according to an embodiment of the present invention.
4 is a view schematically showing the configuration of an output pattern setting unit according to an embodiment of the present invention.
5 is an exemplary view of frame rate adjustment according to an embodiment of the present invention.
6 is a flowchart illustrating a display screen setting method performed by an output pattern setting unit according to an embodiment of the present invention.
7 is a diagram illustrating a process of setting a screen change parameter according to an embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. In the following description of the present invention, if it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. As used herein, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

In addition, the terms “… unit”, “… unit”, “… module” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. Can be.

In addition, in describing the present invention, when it is determined that the detailed description of the related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view schematically showing a configuration of an audio reproducing apparatus according to an embodiment of the present invention, FIG. 2 is a view schematically showing a configuration of an audio reproducing unit according to an embodiment of the present invention, and FIG. 4 is an exemplary view illustrating a display screen according to an embodiment of the present disclosure, FIG. 4 is a view schematically illustrating a configuration of an output pattern setting unit according to an embodiment of the present invention, and FIG. 5 is a frame rate adjustment according to an embodiment of the present invention. It is an illustration about.

Referring to FIG. 1, the audio reproducing apparatus 100 includes an audio reproducing unit 110, an audio output unit 120, a storage unit 130, a communication unit 140, a pattern output unit 150, a display unit 152, The vibration generating unit 154, the output pattern setting unit 160, the input unit 170, and the controller 180 are included. The audio reproducing apparatus 100 according to an embodiment of the present invention may be implemented as any electronic device having, for example, a function of reproducing audio data and outputting a pattern (for example, screen display and / or vibration). Can be.

The audio playback unit 110 selects audio data selected by the user from among audio files stored in the storage 130 and audio streaming data that can be received in real time through the communication unit 140 according to a user's selection of audio data reproduction through the input unit 170. Play it. The audio data may be reproduced by being digitally decoded, for example, PCM data and output to the audio output unit 120.

If the audio file stored in the storage unit 130 is selected and played, the audio reproducing unit 110 may include a file buffer 210 in which an audio file input for reproduction is stored, and a file buffer as illustrated in FIG. 2. A decoder 220 for decoding the audio file stored at 210 and a data buffer 230 in which the decoded audio data is stored.

The audio file stored in the file buffer 210 is, for example, a digital sound source having a format such as MP3 (MPEG Audio Layer 3), WMA (Windows Media Audio), OGG (Ogg Vorbis), and the like. And a body including a plurality of frames and a tail representing the end of the file. The header of the audio file includes information such as file type, version, sample rate, number of samples per channel, compression information, number of required bits, and free format, so that the decoder 220 may provide control information for decoding. The frame constituting the body is a minimum unit for reproducing the audio file, and the audio file stored in the file buffer 210 may be provided to the decoder 220 in units of frames.

The decoder 220 converts an audio file stored in the file buffer 210 to generate audio data. Here, the audio data may be generated in the form of PCM data which is widely used in decoding techniques such as voice and music. The decoder 220 may be provided with at least one decoder, for example, an MP3 decoder, a WMA decoder, an OGG decoder, or the like, depending on the format of the audio file.

The data buffer 230 stores the audio data converted by the decoder 220. The data buffer 230 may sequentially accumulate and store audio data decoded in units of frames.

In addition, when the audio data selected by the user is not a digitally encoded file, the digital data such as PCM data may be decoded through an analog-to-digital converter (not shown) and output to the audio output unit 120.

The audio output unit 120 converts the digital audio data provided from the audio reproducing unit 110 into an analog audio signal and transmits the same to an user through an audio output device such as a speaker (not shown) or earphone (not shown). . The audio output unit 120 may be a codec that performs digital-analog convert in a predetermined manner.

For example, the storage unit 130 may be provided from an external electronic device through an interface such as a USB, or may be downloaded through an audio file or audio streaming data or a pattern output unit 150 downloaded through a communication unit 140 connected to a wired or wireless Internet. Output pattern setting information for setting an output pattern to be output can be stored.

If the output pattern is a display screen, object property parameters for one or more dynamic objects included in the display screen displayed through the display unit 152, a screen change parameter for changing the background or / and dynamic objects of the display screen, One or more of a speed and / or a speed parameter indicating a virtual playback speed of an audio frame may be further stored in the storage 130.

When the output pattern is vibration, a vibration parameter representing the intensity or / and pattern of vibration generated through the vibration generator 154 may be further stored in the storage 130.

The storage unit 130 may be, for example, a nonvolatile memory such as NAND FLASH, NOR FLASH, SD CARD, MMC CARD, CF CARD, or the like.

The communicator 140 is communicatively connected to other electronic devices through a network such as a wired or wireless Internet to receive audio data including at least one of an audio file and audio streaming data. In particular, in the case of audio streaming data, a user's request signal may be transmitted, and audio streaming data corresponding to the request signal may be received to be reproduced in real time by the audio reproducing unit 110.

The pattern output unit 150 reproduces the current audio output unit 110 and outputs the audio data through the audio output unit 120 according to the output pattern setting information set by the output pattern setting unit 160 to be described later. Make sure that output patterns that match (characteristics) are printed. In the present embodiment, it is assumed that the pattern output unit 150 is provided in the audio reproducing apparatus 100. However, the scope of the present invention is not limited thereto, and a pattern output such as a display device and / or a vibration generating device is described. The device may be externally connected.

The pattern output unit 150 may include a display unit 152 or a vibration generator 154 depending on the target to which the output pattern is applied.

In one embodiment, the output pattern is located on the background 310, which is a background image, and superimposed on the background 310 as illustrated in FIG. 3, but independently of its position, size, rotation amount, speed, etc. One or more dynamic objects 320a, 320b, 320c (hereinafter, collectively referred to as “320”) implemented to be changeable may be applied to the display screen 300 configured and may be displayed through the display unit 152.

The display screen 300 includes a background 310 and a dynamic object 320 having various types and colors, and the background 310 and the dynamic object 320 itself are changed or the position, size, and location of the dynamic object 320 are changed. One or more of the amount of rotation, speed, etc. is variable, so that the display screen 300 corresponding to the characteristics of the audio data, for example, the speed of the music being played, the genre, etc. is output through the display unit 152 to induce the user's interest. And interest persistence may be possible.

The dynamic object 320 may include a first dynamic object 320a having a one-dimensional or two-dimensional pattern including dots, lines, and surfaces, and a second dynamic object in which a photo image stored in the storage unit 130 is displayed. 320b, a third dynamic object 320c having a three-dimensional shape through three-dimensional modeling may be provided.

Such dynamic objects may express the characteristics of the audio data by changing their size, rotating around a predetermined central axis penetrating the dynamic object, or changing their position or moving speed within the display screen 300.

The initial display screen may be set as a default or may be set by a user's selection through the input unit 170.

Alternatively, in another embodiment, the output pattern is applied to a vibration capable of controlling one or more of the intensity and the pattern, and the vibration may be generated through the vibration generator 154.

The output pattern setting unit 160 generates and outputs output pattern setting information in which an output pattern is set according to the characteristics of the corresponding audio data according to a result of analyzing the audio data output from the audio reproducing unit 110.

In the case of audio data, when the sound volume is not leveled to a constant level, there is a problem in that the variation of the analysis result in the output pattern setting unit 160 is severe. Accordingly, the output pattern setting unit 160 further includes a leveling module (not shown) to reduce the deviation by leveling the volume to a predetermined level before analyzing the audio data and setting the output pattern according to the result. have.

As illustrated in FIG. 4, the output pattern setting unit 160 includes an audio frame dividing unit 410, a frame level calculating unit 420, and an output pattern control unit 430. According to an embodiment, the apparatus may further include one or more of the object characteristic parameter setting unit 440, the screen change parameter setting unit 450, the speed parameter setting unit 460, and the vibration parameter setting unit 470.

The audio frame dividing unit 410 receives audio data being reproduced for sound source reproduction from the audio reproducing unit 110 and divides the audio data into one or more audio frames. An audio frame corresponding to a processing unit includes one or more audio samples.

The size of the audio frame may be determined according to the output capability of the pattern output unit 150. The output capability is the resolution ability of the output pattern that the pattern output unit 150 can express. For example, when the pattern output unit 150 is the display unit 152, fps (frame per second) is the number of screen updates per second. ) Corresponds to this, and when the pattern output unit 150 is the vibration generator 154, the number of vibration state updates per second corresponds to this.

According to this, for example, the size of an audio frame corresponding to the display unit 152 having an output capability of 30 fps or the vibration generator 154 that can change the vibration state 30 times per second may be about 33 ms (= 1000 ms / 30 fps). .

According to an exemplary embodiment, the audio frame may be the same as the frame constituting the body of the audio file corresponding to the minimum unit processed by the audio reproducing unit 110 when the audio file is reproduced. In this case, the audio frame dividing unit 410 may utilize each frame constituting the body of the audio file as an audio frame.

The frame level calculator 420 calculates a frame level for each divided audio frame. The frame level is a level average of one or more audio samples belonging to the corresponding audio frame. For example, when each audio frame includes 50 audio samples, the average value for each level of the 50 audio samples becomes the frame level. Here, the audio data may be PCM data as described above, and the level of the audio sample may be a PCM level indicating one of the quantization levels.

In addition, the frame level calculator 420 may calculate a frame level for each frequency band according to an embodiment. When the output pattern is applied to the vibration, Fourier transform may be performed on the audio data to calculate the frame level for each frequency band.

The output pattern controller 430 sets an output pattern to which an output pattern to which a parameter set (one or more of an object characteristic parameter, a screen change parameter, a speed parameter, and a vibration parameter) set corresponding to the frame level calculated by the frame level calculator 420 is applied. Generate and print the setting information.

First, a case in which an output pattern is applied to the display screen 300 will be described.

In this case, the output pattern setting information includes background information on the type and color of the background 310 and dynamic object information on the type, color, position, size, rotation amount, and speed of the dynamic object 320. As parameters for changing one or more of the background information and the dynamic object information, object property parameters that control one or more of the size and amount of rotation of the dynamic object 320, the type and color of the background 310, the dynamic object 320 ), A screen change parameter for adjusting one or more of a kind, color, and position, a speed parameter for adjusting one or more of the speed and frame rate of the dynamic object 320.

The object property parameter setting unit 440 sets an object property parameter for controlling the property of the dynamic object included in the display screen corresponding to the frame level calculated by the frame level calculator 420. Here, the corresponding setting is not only determined proportionally or inversely, but also includes determining the same value, and the proportional and inverse relation is not limited to the linear one.

Object properties may include one or more of the size and amount of rotation of the dynamic object. Therefore, the object characteristic parameter is used as a variable for controlling one or more of the size and rotation amount of the dynamic object in the display screen output through the display unit 152. Here, the size of the dynamic object is a concept including one or more of the length, height, width, volume, etc. of the dynamic object, and the amount of rotation of the dynamic object is about a predetermined central axis penetrating the dynamic object. Indicates the degree of rotation in the forward or reverse direction.

One object property parameter may be collectively applied to all one or more dynamic objects included in the display screen, or as required, separate object property parameters may be allocated and individually applied to each object group including one or more dynamic objects. .

The screen change parameter setting unit 450 sets a screen change parameter for changing to a new display screen corresponding to a time point when a predetermined time elapses from the previous screen change point and the frame level of the audio frame to be determined becomes equal to or greater than a reference value. .

The change of the screen means that one or more of the background 310 and the dynamic object 320 constituting the display screen 300 as illustrated in FIG. 3 is changed, and the background 310 or / and the dynamic object 320 is changed. It may mean that one or more of the type and color of the background 310 are changed or one or more of the type, color, and position of the dynamic object 320 are changed.

Here, the reference value to be compared for changing the screen may be a value in which the reference constant is converted to correspond to an average of frame levels for audio frames belonging to a predetermined time unit. Here, the corresponding transformation is not only determined proportionally or inversely, but also includes determining the same value, and the proportional and inverse relation is not limited to being linear. Here, the reference constant is a constant related to the frequency of the screen change point, and may be a real number equal to or greater than zero (zero) and one or less. The reference constant is determined by default or can be changed by the user.

For example, a reference constant for determining a reference value is set to 0.7, and the frame level average of audio frames belonging to a certain time unit (for example, 10 seconds) belonging to the first section is 50, and the past belonging to the second section is 50. Assume that the frame level average of audio frames belonging to a certain time unit is 60. In this case, 70 (= 0.7 * 50 * 2) is used as a reference value for audio frames input after the first interval, and 84 (= 0.7 * 60 * 2) is used as reference values for audio frames input after the second interval. Can be used.

The speed parameter setting unit 460 sets a speed parameter for adjusting the speed or / and the frame rate of the dynamic object in the display screen corresponding to the average of the frame levels for the audio frames belonging to a certain time unit in the past.

The speed of the dynamic object is the positional displacement of the dynamic object per hour, and the frame rate is the virtual playback speed of audio data which is a reference for setting the display screen. The value can be determined. Herein, the value to be determined correspondingly means not only to be determined proportionally or inversely but also to be determined to be the same value, and the proportional and inverse relationship is not limited to the linear one.

The frame rate is distinguished from the reproduction rate of the audio data reproduced by the audio reproducing unit 110 and output to the audio output unit 120 and provided as auditory information.

Referring to FIG. 5, an example of a process of adjusting a frame rate is shown.

FIG. 5A illustrates audio data 500 for reproducing sound source reproduced by the audio reproducing unit 110 and output to the audio output unit 120 to be converted into an analog audio signal. The time stamps corresponding to the first frame F1, the second frame F2, the third frame F3, and the fourth frame F4 are set to 0, 100, 200, and 300, respectively. Same as 100.

FIG. 5B shows the display screen setting audio data 510 which is used as the purpose of setting the display screen in the output pattern setting unit 160. As described above, the frame rate of each audio frame may be adjusted according to the speed parameter set by the speed parameter setting unit 460.

In the display data setting audio data 510, as illustrated in FIG. 5B, the first audio frame F1 ′, the second audio frame F2 ′, and the third audio frame F3 correspond to the speed parameter. ') And the time stamps corresponding to the fourth audio frame F4' may be set to 0, 100, 190, and 310, respectively.

That is, the output pattern setting unit 160 determines that the audio data currently being reproduced is reproduced together with the display screen setting audio data 510 and configures the display screen accordingly. That is, the display unit 152 may display a display screen that changes rapidly or slowly according to each audio frame that is virtually reproduced quickly or slowly in the display screen setting audio data 510.

Referring to the audio data illustrated in FIGS. 5A and 5B, the first audio frame F1 ′ occupies the same section as the first frame F1 and has a normal playback speed. The second audio frame F2 'and the fourth audio frame F4' occupy a 10% smaller section than the second frame F2 and the fourth frame F4, respectively, and thus have a relatively high playback speed. The three audio frames F3 'occupy a 20% larger section than the third frame F3, and thus have a relatively slow playback speed. As such, the change speed of the background and the dynamic object included in the display screen is also changed to correspond to the variable frame rate, thereby enabling the display screen to be more suitable for the characteristics of the audio data.

Next, in another embodiment, a case in which the output pattern is applied to vibration will be described.

In this case, the output pattern setting information includes vibration information on at least one of vibration intensity and pattern.

The vibration parameter setting unit 470 sets a parameter for changing the vibration information. As a parameter for changing the vibration information, it is used as a variable that adjusts one or more of the intensity and pattern of vibration according to the frame level for each frequency band for one or more frequency bands selected by default or by a user. Here, one or more of the intensity and pattern of the vibration may be applied corresponding to the frame level value of the corresponding frequency band. Herein, the corresponding application is not only applied proportionally or inversely. It is meant to include being applied with the same value, and the proportional and inverse relationship is not limited to the linear one.

In addition, the aforementioned object characteristic parameters, screen change parameters, speed parameters, and the like may also be used as parameters for changing vibration information.

For example, a vibration may be generated to identify a time point at which a display screen is changed according to audio data currently being reproduced. The time point at which the display screen is changed is controlled by the screen change parameter described above, and thus the vibration generator 154 may be controlled to generate a vibration based on the screen change parameter.

Next, the vibration may be generated by applying one or more of the intensity and the pattern differently according to the frame level of the audio data currently being reproduced. The information on the frame level of each audio frame may be checked from the aforementioned object characteristic parameter, and the vibration generator 154 may be controlled to generate vibration based on the information about the frame level of each audio frame.

Referring back to FIG. 1, the display unit 152 outputs the state set in the output pattern setting unit 160 corresponding to the state of the audio reproducing apparatus 100, the operation mode, the information of the audio data being reproduced, and the audio data selected by the user. Image data corresponding to the display screen generated according to the pattern setting information is displayed.

The vibration generator 154 generates a vibration in which one or more of its intensity and pattern are set according to the output pattern setting information, and different vibrations are synthesized and output or sequentially output.

The vibration generating unit 154 may be implemented as a vibration device for mechanically vibrating the audio reproducing apparatus 100, and has a size and performance suitable for embedding in the audio reproducing apparatus 100 among various commercially available products. Can be selected.

The vibration generator 154 may be controlled to generate vibration by varying one or more of intensity and pattern according to parameters or output pattern setting information set by the output pattern setting unit 160.

The input unit 170 is a means for receiving manipulation information by a user, and may be implemented as one or more buttons. If the display unit 152 is implemented as a touch sensitive screen, the display unit 152 and the input unit 170 may be integrated.

The controller 180 controls the operation of each component of the audio reproduction apparatus 100 described above.

6 is a flowchart illustrating a method of automatically generating an output pattern performed by an output pattern setting unit according to an embodiment of the present invention, and FIG. 7 is a view illustrating a process of setting a screen change parameter according to an embodiment of the present invention. Each step described below may be performed by each internal component of the output pattern setting unit 160.

In operation S600, the audio frame dividing unit 410 is divided into one or more audio frames for audio data output from the audio reproducing unit 110 and transmitted to the user as an analog audio signal through the audio output unit 120. do.

In operation S610, the frame level calculator 420 calculates a frame level for each of the audio frames classified in operation S600. The frame level may be a level average value of one or more audio samples belonging to each audio frame.

In step S620, one or more parameters for setting an output pattern are set based on the calculated frame level. For example, when the output pattern is applied to the display screen, one or more of an object characteristic parameter, a screen change parameter, and a speed parameter may be set.

In operation S620a, the object characteristic parameter setting unit 440 sets an object characteristic parameter for controlling one or more object characteristics among the size and rotation amount of the dynamic object included in the display screen corresponding to the frame level.

In step S620b, the screen change parameter setting unit 450 sets the screen change parameter such that the time when the predetermined time elapses from the previous screen change time and the frame level becomes equal to or greater than the reference value becomes the new screen change time.

Here, the reference value for changing the screen may be converted to correspond to an average of frame levels for audio frames belonging to a certain time unit in the past.

In this case, as shown in FIG. 7, in step S710, the screen change parameter setting unit 450 determines whether a predetermined time has elapsed since the previous screen change time. Here, the predetermined time may be set as a default or determined by a user's input through the input unit 170.

If a predetermined time has not elapsed, the process waits in step S710, and after a predetermined time elapses, the process proceeds to step S720.

In operation S720, the reference value is converted to correspond to an average of frame levels of audio frames belonging to a predetermined time unit on the basis of the reproduction time of the audio frame to be determined.

In operation S730, it is determined whether the frame level is equal to or greater than the converted reference value. If the frame level is less than the converted reference value, the process returns to step S720 to repeat the reference value conversion on successive audio frames. Set the screen change parameters.

Referring back to FIG. 6, in step S620c, the speed parameter setting unit 460 corresponds to an average of frame levels of audio frames belonging to a predetermined time unit based on a playback time point of an audio frame to be determined. Set the speed parameter to adjust the speed or frame rate.

The above-described steps S620a, S620b, and S620c may be performed independently and may be performed sequentially or nonsequentially.

In operation S630, the output pattern controller 430 generates and outputs output pattern setting information to which one or more of an object characteristic parameter, a screen change parameter, and a speed parameter are applied.

According to an embodiment, step S640 of controlling the vibration generating unit 154 to generate vibration based on one or more of an object characteristic parameter, a screen change parameter, a speed parameter, and / or output pattern setting information may be further performed. . For example, vibration may be generated to identify a time point at which the display screen is changed, or vibration may be generated in which one or more of intensity and pattern are differently applied according to the frame level of the audio frame currently being played.

In addition, after calculating the frequency band level by performing Fourier transform on the audio data, vibration is generated when the level of each frequency band calculated for one or more frequency bands designated by default or selected by the user is greater than or equal to a predetermined threshold value. You can also make it work.

In addition, before performing the output pattern automatic generation process according to the present embodiment, the original audio data by leveling the volume of one or more audio data stored in the storage 130 or received through the communication unit 140 to a predetermined level. There is an advantage to reduce the deviation of the analysis results due to the volume difference irrespective of the nature existing between.

It is apparent that the above-described method for automatically generating the output pattern may be performed by an automated procedure according to a time series sequence by a software program or the like embedded in the audio reproducing apparatus. The codes and code segments that make up the program can be easily deduced by a computer programmer in the field. The program is also stored in a computer readable information storage medium, and read and executed by the computer to implement the method. The information storage medium includes a magnetic recording medium, an optical recording medium, and a carrier wave medium.

Although the above has been described with reference to embodiments of the present invention, those skilled in the art may variously modify the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. And can be changed.

100: audio playback device 110: audio playback unit
120: audio output unit 130: storage unit
140: communication unit 150: pattern output unit
152: display unit 154: vibration generating unit
160: output pattern setting unit 170: input unit
180: control unit 300: display screen
310: Background 320a, 320b, 320c: Dynamic Object
410: audio frame divider 420: frame level calculator
430: output pattern control unit 440: object characteristic parameter setting unit
450: screen change parameter setting unit 460: speed parameter setting unit
470: vibration parameter setting unit

Claims (27)

A device for automatically generating an output pattern that can be output through an output device in accordance with audio data being reproduced.
An audio frame dividing unit which receives the audio data being reproduced in the sound source and divides the audio data into one or more audio frames, wherein the size of the audio frame has a size determined according to an output capability of the output device;
A frame level calculator for calculating a frame level for each of the audio frames; And
And an output pattern controller configured to generate an output pattern corresponding to output pattern setting information set according to a parameter set according to the frame level.
The audio data is a digital signal decodable according to a pulse code modulation (PCM) scheme, wherein the frame level is an average of PCM levels of one or more audio samples belonging to each of the audio frames.
The method of claim 1,
The output device is a display device,
And the output pattern is applied to a display screen on which a background is determined according to the output pattern setting information, at least one dynamic object is disposed, and is displayed on the display device.
The method of claim 2,
The input audio data functions as audio data for display screen setting.
The parameter is a speed parameter for adjusting a frame rate, which is a virtual reproduction speed of the display screen setting audio data as a reference for setting the output pattern,
And a speed parameter setting unit configured to set the speed parameter by adjusting a time stamp of the one or more audio frames included in the display screen setting audio data corresponding to an average of frame levels of audio frames belonging to a certain time unit in the past. Automatic output pattern generating device comprising a.
delete The method of claim 2,
The parameter is an object property parameter that controls one or more of the size and amount of rotation of the dynamic object,
And an object characteristic parameter setting unit for setting the object characteristic parameter corresponding to the frame level.
The method of claim 2,
The parameter is a screen change parameter relating to a point in time at which the display screen is changed;
A screen change parameter setting unit is further configured to set the screen change parameter for determining a point in time at which the display screen is changed when a predetermined time elapses from a previous screen change point and a frame level of an audio frame to be determined is equal to or greater than a reference value. Automatic output pattern generating device comprising a.
The method of claim 6,
The reference value is an automatic generation of an output pattern, characterized in that a reference constant related to the frequency of the screen change time is converted to correspond to an average of the frame levels of audio frames belonging to a predetermined time unit based on the audio frame to be determined. Device.
The method of claim 2,
The parameter is a speed parameter for adjusting the speed of the dynamic object,
And a speed parameter setting unit for setting the speed parameter corresponding to an average of frame levels of audio frames belonging to a certain time unit in the past.
The method of claim 8,
And the speed of the dynamic object is a positional displacement of the dynamic object per hour.
delete The method of claim 1,
The output device is a vibration generating device,
And the output pattern is applied to vibration generated by the vibration generating device by varying one or more of intensity and pattern according to the output pattern setting information.
The method of claim 11,
The frame level calculator performs Fourier transform on the audio data to calculate a frequency band level.
The output pattern setting unit generates the output pattern setting information for generating the vibration when the frequency band-specific level calculated for the one or more frequency bands designated by default or selected by the user is greater than or equal to a predetermined threshold value. Automatic output pattern generation device.
The method of claim 12,
And the output pattern setting unit generates the output pattern setting information for applying at least one of the vibration intensity and the pattern corresponding to the frequency band level.
A method for automatically generating an output pattern in an output pattern generation apparatus provided in an audio reproducing apparatus and outputting an output pattern through an output apparatus so as to correspond to the characteristics of the audio data reproduced in the audio reproducing apparatus.
(a) dividing the audio data selected by the user into one or more audio frames, wherein the size of the audio frame has a size determined according to the output capability of the output device;
(b) calculating a frame level for each of the audio frames; And
(c) setting and outputting output pattern setting information according to a parameter set according to the frame level,
An output pattern corresponding to the output pattern setting information is output through the output device,
The audio data is a digital signal decodable according to a pulse code modulation (PCM) scheme, and the frame level is an average of PCM levels of one or more audio samples belonging to each of the audio frames.
The method of claim 14,
The output device is a display device,
And the output pattern is applied to a display screen on which a background is determined according to the output pattern setting information, at least one dynamic object is disposed, and is displayed on the display device.
16. The method of claim 15,
The audio data selected by the user functions as audio data for display screen setting,
The parameter is a speed parameter for adjusting a frame rate, which is a virtual reproduction speed of the display screen setting audio data as a reference for setting the output pattern,
Prior to the step (c), the speed parameter is adjusted by adjusting a time stamp of the one or more audio frames included in the display screen setting audio data corresponding to an average of frame levels for audio frames belonging to a certain time unit in the past. Automatically generating the output pattern, characterized in that it further comprises the step of setting.
delete 16. The method of claim 15,
The parameter is an object property parameter that controls one or more of the size and amount of rotation of the dynamic object,
Before the step (c), further comprising the step of setting the object characteristic parameter corresponding to the frame level.
16. The method of claim 15,
The parameter is a screen change parameter relating to a point in time at which the display screen is changed;
Before the step (c), a predetermined time elapses from the previous screen change time point, and the screen change parameter for determining the time when the frame level of the audio frame to be determined is equal to or more than the reference value is the time point when the display screen is changed. Automatically generating the output pattern, characterized in that it further comprises the step of setting.
The method of claim 19,
The reference value is an automatic generation of an output pattern, characterized in that a reference constant related to the frequency of the screen change time is converted to correspond to an average of the frame levels of audio frames belonging to a predetermined time unit based on the audio frame to be determined. Way.
16. The method of claim 15,
The parameter is a speed parameter for adjusting the speed of the dynamic object,
Before the step (c), further comprising the step of setting the speed parameter corresponding to the average of the frame level for audio frames belonging to a certain time unit in the past.
The method of claim 21,
And a velocity of the dynamic object is a time displacement of the dynamic object.
delete The method of claim 14,
The output device is a vibration generating device,
The output pattern is automatically generated output pattern, characterized in that applied to the vibration generated by the vibration generating device by varying one or more of the intensity and pattern according to the output pattern setting information.
25. The method of claim 24,
Comprising a step of performing the Fourier transform on the audio data before step (c) to calculate the level for each frequency band,
The step (c) is to automatically generate the output pattern, characterized in that for generating the vibration when the frequency band-specific level calculated for the one or more frequency band specified by the default or more than a predetermined threshold value.
The method of claim 14,
Prior to the step (a), the step of equalizing the volume of the audio data to a predetermined level is performed automatically.
A program of instructions that can be executed by a digital processing apparatus to perform the method for automatically generating the output pattern according to any one of claims 14 to 16, 18 to 22, and 24 to 26 is tangible. A recording medium having a program recorded thereon which can be read by a digital processing device.

Images