KR20190009821A - Method and system for generating playlist using sound source content and meta information - Google Patents

Abstract

A method for automatically generating playlists using sound source content and meta information and a system thereof are disclosed. The method of generating playlists includes: a step of generating acoustic features unique to a sound source from the sound source data of a sound source content for each sound source content; and a step of generating a playlist based on similarity in the sound source content by calculating the similarity in the sound source content using the acoustic features.

Description

[0001] METHOD AND SYSTEM FOR GENERATING PLAYLIST USING SOUND SOURCE CONTENT AND META INFORMATION [

The following description relates to a technique for automatically generating a playlist for a sound source.

Recently, people have routinely used media content such as music pieces, video or audio tracks on various types of media. A playlist provides users with the ability to organize music or other content based on preferences, themes, genres, artists or occasions.

Korean Patent No. 10-0810276 (Feb. 27, 2008) discloses a technique for creating a user play list for a plurality of sound source data stored in a sound source data reproducing apparatus as an example of a play list generating technique .

In the past, a playlist is generated from a sound source having a playback history or a preview history, or a playlist is generated by a similarity measurement method using a collaborative filtering feature. Since the existing playlist generation method does not use the characteristics of the sound source itself as a factor, there is a high probability that a song having a characteristic different from that of the start song (automatic play) during automatic playback is obtained. In addition, when the playlist is generated by using the metadata of the sound source as a strong filter condition, there is a problem that the diversity of the playlist is deteriorated.

There is provided a method and system for automatically generating a playlist by selecting a song that is similar to a song having a user's favorite song or a playback history using a model in which meta information of a sound source and a sound source is learned.

A playlist generation method performed in a computer system, the method comprising: generating acoustic features unique to a sound source from sound source data of the sound source content for each sound source content; And generating a playlist based on the degree of similarity between the sound source contents by calculating a degree of similarity between the sound source contents using the acoustic feature, A play list similar to the current play song is selected to generate a play list. When a next play song is selected, the similarity with the seed song is checked, And generating a play list based on the generated play list.

There is provided a computer-readable recording medium having recorded thereon a program for causing the computer to execute the playlist generating method.

A computer-implemented playlist generating system, comprising: a unique feature processor for generating a unique acoustic feature of a corresponding sound source from sound source data of the sound source content for each sound source content; And a playlist generating unit for calculating a similarity between the sound source contents using the acoustic feature and generating a play list based on the similarity between the sound source contents, wherein the play list generating unit generates a play list based on the similarity between the sound source contents, And generating a play list in a chain form from sound source contents similar to the seed song by checking the similarity between the next play song and a seed song when a next play song is selected, Provides a list generation system.

According to the embodiment of the present invention, by using deep learning, eigen-value of the sound source itself is used as a factor for learning, so that the similarity between songs can be more accurately calculated.

According to the embodiment of the present invention, it is possible to calculate the similarity between songs by using the meta information of the sound source as additional information as well as the characteristics of the sound source, and to apply the logic to continuously check the similarity with the seed song So that it is possible to prevent the playlist from being included in a song that is too different from the seed song while maintaining the diversity of the play list.

1 is a block diagram for explaining an example of the internal configuration of a computer system according to an embodiment of the present invention.
2 is a diagram illustrating an example of components that a processor of a computer system according to an embodiment of the present invention may include.
3 is a flowchart illustrating an example of a play list generation method that can be performed by a computer system according to an embodiment of the present invention.
4 is a diagram illustrating an example of a process of automatically generating a playlist according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The present embodiments relate to methods and systems for generating playlists, and more particularly to a playlist generating technique capable of automatically generating playlists using a model in which meta information of sound sources and sound sources is learned .

Embodiments, including those specifically disclosed herein, achieve the creation of playlists for source content and thereby achieve significant advantages in terms of cost savings, efficiency and accuracy, and the like.

The play list generated for the sound source content can be utilized for recommending, searching, classifying, and managing the sound source content. For example, in order to recommend a user-customized sound source, a sorting operation for songs similar to the user's favorite music is required. As a sorting operation, a playlist can be automatically generated using meta information of a sound source and a sound source.

1 is a block diagram for explaining an example of the internal configuration of a computer system according to an embodiment of the present invention. For example, a playlist generation system in accordance with embodiments of the present invention may be implemented through the computer system 100 of FIG. 1, the computer system 100 includes a processor 110, a memory 120, a persistent storage 130, a bus 140, an input / output interface 150 and a network interface 160.

Processor 110 may include or be part of any device capable of processing any sequence of instructions. The processor 110 may comprise, for example, a processor and / or a digital processor within a computer processor, a mobile device, or other electronic device. The processor 110 may be, for example, a server computing device, a server computer, a series of server computers, a server farm, a cloud computer, a content platform, a mobile computing device, a smart phone, a tablet, a set top box, The processor 110 may be connected to the memory 120 via a bus 140.

The memory 120 may include volatile memory, permanent, virtual or other memory for storing information used by or output by the computer system 100. Memory 120 may include, for example, random access memory (RAM) and / or dynamic RAM (DRAM). The memory 120 may be used to store any information, such as the state information of the computer system 100. Memory 120 may also be used to store instructions of computer system 100 including, for example, instructions for generating playlists for tone generator content. Computer system 100 may include one or more processors 110 as needed or where appropriate.

The bus 140 may comprise a communication infrastructure that enables interaction between the various components of the computer system 100. The bus 140 may, for example, carry data between components of the computer system 100, for example, between the processor 110 and the memory 120. The bus 140 may comprise a wireless and / or wired communication medium between the components of the computer system 100 and may include parallel, serial, or other topology arrangements.

The persistent storage device 130 may be a component such as a memory or other persistent storage device as used by the computer system 100 to store data for a predetermined extended period of time (e.g., as compared to the memory 120) Lt; / RTI > The persistent storage device 130 may include non-volatile main memory as used by the processor 110 in the computer system 100. The persistent storage device 130 may include, for example, flash memory, hard disk, optical disk, or other computer readable medium.

The input / output interface 150 may include a keyboard, a mouse, voice command inputs, displays, or interfaces to other input or output devices. Configuration commands and / or inputs for generating a playlist, such as sound source data and text information, may be received via the input / output interface 150.

The network interface 160 may include one or more interfaces to networks such as a local area network or the Internet. The network interface 160 may include interfaces for wired or wireless connections. The sound source data and the text information of the sound source contents for constitutional commands and / or play list generation can be received via the network interface 160.

Also, in other embodiments, the computer system 100 may include more components than the components of FIG. However, there is no need to clearly illustrate most prior art components. For example, the computer system 100 may be implemented to include at least some of the input / output devices connected to the input / output interface 150 described above, or may include a transceiver, a Global Positioning System (GPS) module, Databases, and the like. More specifically, when the computer system 100 is implemented in the form of a mobile device such as a smart phone, an acceleration sensor, a gyro sensor, a camera, various physical buttons, buttons using a touch panel, An input / output port, a vibrator for vibration, and the like may be further included in the computer system 100.

FIG. 2 is a diagram illustrating an example of a component that a processor of a computer system according to an exemplary embodiment of the present invention may include; FIG. 3 is a block diagram of a playlist Fig. 8 is a flowchart showing an example of a generation method. Fig.

2, the processor 110 may include a sound source input control unit 210, a sound source preprocessing unit 220, an intrinsic feature processing unit 230, and a play list generation unit 240. The components of such a processor 110 may be representations of different functions performed by the processor 110 in accordance with control commands provided by at least one program code. For example, the sound source input control 210 may be used as a functional representation in which the processor 110 operates to control the computer system 100 to receive sound source data. The components of the processor 110 and the processor 110 may perform the steps S310 to S350 included in the method of generating a playlist of FIG. For example, the components of processor 110 and processor 110 may be implemented to execute instructions in accordance with the at least one program code described above and the code of the operating system that memory 120 contains. Wherein at least one program code may correspond to a code of a program implemented to process the playlist generation method.

The method of generating a playlist may not occur in the order shown, and some of the steps may be omitted or an additional process may be further included.

In step S310, the processor 110 may load the program code stored in the program file for the method of generating a play list into the memory 120. [ For example, a program file for the method of generating a playlist may be stored in the persistent storage 130 described above with reference to FIG. 1, and the processor 110 may store the program files stored in the persistent storage 130, The computer system 110 may be controlled such that the program code is loaded into the memory 120. [

At this time, each of the sound source input control unit 210, the sound source preprocessing unit 220, the inherent feature processing unit 230, and the play list generation unit 240 included in the processor 110 and the processor 110 is stored in the memory 120 And may be different functional representations of the processor 110 for executing instructions of a corresponding one of the loaded program codes to execute subsequent steps S320 through S350. For the execution of steps S320 through S350, the processor 110 and the components of the processor 110 may process an operation according to a direct control command or control the computer system 100. [

In step S320, the sound source input control unit 210 may control the computer system 100 to receive the sound source data of the sound source content with respect to the sound source content. In this case, the sound source content may refer to all digital data having an audio file format, and may include, for example, MPEG Audio Layer-3 (MP3), Waveform Audio Format (WAVE), Free Lossless Audio Codec have. The sound source input control unit 210 may control the computer system 100 to receive text information related to the sound source content. The text information related to the sound source content may include lyrics, meta information such as a singer, a genre, a title, an album name, etc., and a hash tag (hashtag), a query Information. The sound source data and text information may be input to or received from the computer system 100 via the input / output interface 150 or the network interface 160 and stored and managed in the memory 120 or the permanent storage 130.

In step S330, the sound source preprocessing unit 220 may process the sound source data received in step S320 in the form of learning data. At this time, the sound source preprocessing unit 220 can express the sound source data in a time-frequency manner through the preprocessing. For example, the sound source preprocessing unit 220 may convert the sound source data into data of a time-frequency-size type such as a Mel-spectrogram or a Mel Frequency Cepstral Coefficient (MFCC). The sound source preprocessing unit 220 can preprocess the text information when the text information is inputted together with the sound source content. For example, the sound source preprocessing unit 220 can filter meaningless texts from input text information using a language preprocessor such as a morpheme analyzer and an indexer extractor. In other words, the sound source preprocessing unit 220 removes unnecessary parts of speech words and special symbols (!,?, /, Etc.) included in the text information and extracts words corresponding to the cognition or the root can do.

In step S340, the intrinsic feature processing unit 230 may learn the preprocessed training data on the sound source data through the learning model to generate the intrinsic feature, and store the intrinsic feature in the database (not shown). The intrinsic feature processing unit 230 can use the deep run to generate unique acoustic features of the sound source data itself. For example, the intrinsic feature processor 230 may use a CNN (Convolutional Neural Network) based learning model. The intrinsic feature processor 230 can express the sound source data as a multidimensional real number vector using the CNN learning model. The CNN learning model may include a sound source data learning layer. Steps 1 to 3 below may be an example of a process of generating a real vector corresponding to sound source data in the sound source data learning layer.

In step 1, the sound source data (for example, an mp3 file) included in the sound source content can be converted into data of a time-frequency-size type such as a mel-spectrogram or MFCC through preprocessing. For example, the sound source preprocessing unit 220 may represent the sound source data input by the sound source input control unit 210 in time-frequency.

A plurality of frequency frames for one or more short time intervals (1 second to 10 seconds) may be sampled from the sound source data converted in step 2 and used as input data for the learning model of the sound source data. For example, the intrinsic feature processor 230 may sample a plurality of frames and utilize the sampled frames as an input to a CNN model presented as an example of a sound source model in a sound source data learning layer. Therefore, the CNN model for learning sound source data can be a model having the same number of channels as the number of sampled frames. Alternatively, each generated frame may be used as a single channel, each frame may be subjected to a specific convolution / pooling process, and then the generated feature vectors may be jointly used as an input of a complete connection layer.

In step 3, a plurality of convolution and pooling layers, which may be included in the sound source data learning layer, are repeatedly generated, thereby generating abstract features from sound source frames. The size of the patches in the convolution can be variously configured, and the pooling method using the pooling maximum value (max), the pooling method using the average value (average), and the hybrid pooling method combining the two pooling methods At least one can be used.

There are fully-connected layers for creating acoustic features on the plurality of convolution and pooling layers, and each layer function includes a sigmoid function, a hyperbolic tangent (tanh) function, a ReLU Rectified Linear Unit) function can be used. As a result, one multi-dimensional real vector can be generated for the sound source data. For example, given the first source data m0, x0 = {0.2, -0.1, 0.3, ... } Can be represented by one multidimensional real number vector. The number of dimensions is an integer greater than 0 and can have a value of typically not less than 50 and not more than 500 empirically. This process 3 can be processed by the inherent feature processor 230 described above.

If the intrinsic feature processor 230 receives the text information together with the sound source content, the intrinsic feature processor 230 may also express the text information as a multidimensional real number vector. For example, the intrinsic feature processing unit 230 may generate a word vector by using a pre-learned learning model through a preprocessing of text information. For example, a word vector may be represented as a numerical multidimensional vector form. Word frequency histogram, TF (term frequency) / IDF (inverse document frequency), language learning model (e.g., word2vec, phrase2vec, document2vec, etc.) can be used for word vector generation. For example, Singer / Genre / Title / Year is one n-dimensional real number vector v = {0.3, -1.2, 1.2, ... }. In addition, the order of the text information field (singer, genre, title, year, etc.) for the language learning model is not fixed, but can be changed according to the purpose.

The intrinsic feature processor 230 may store and maintain the feature vectors of the sound source data and the word vectors of the text information for the sound source content in the database. The feature vectors for the sound source data and the word vectors for the text information may be constructed as separate databases, respectively, or as a single database. It is also possible that such a database is implemented as an element contained in the computer system 100 or as an external database built on a separate system capable of interfacing with the computer system 100. [

In step S350, the play list generation unit 240 may calculate the similarity between the sound sources using the unique characteristics of the sound source data stored in the database with respect to the sound source contents. Based on the similarity between the sound sources, Can be automatically generated. For example, the playlist generation unit 240 may calculate similarities between sound sources using feature vectors of sound source data. In another example, the playlist generation unit 240 may use a word vector for text information in combination with a feature vector for sound source data The similarity between sound sources can be calculated. At this time, the play list generating unit 240 can infinitely generate a play list in the form of a chain of songs similar to the seed song, using the song as a seed song when the user selects a specific song. The play list generating unit 240 may generate a play list based on the user's recent log history of the sound source content, and if a specific singer is given as an input, It is also possible to create playlists.

Accordingly, the present invention can automatically generate playlists of similar songs based on the similarity between songs using a model in which text information such as sound sources and meta information of sound sources are learned.

4 is a diagram illustrating an example of a process of automatically generating a playlist according to an embodiment of the present invention. Steps S401 to S407 of FIG. 4 may be performed in step S350 of FIG.

In step S401, when the seed song is selected, the play list generating unit 240 may calculate the similarity between the seed song and the other songs using the unique characteristics of each sound source stored in the database. For example, when a user selects and reproduces a specific song, the song can be selected as a seed song. As another example, a user's favorite song can be selected as a seed song based on the user's recent log history of the sound source content. For example, when the seed song is m0, the processor 110 uses the CNN model to convert the seed song m0 to a d-dimensional real number vector x0 representing the unique characteristics of the sound source itself, when the seed song m0 is a new song not on the database If the seed song m0 is not a new song, a d-dimensional real number vector corresponding to the seed song m0 is retrieved from the database and defined as x0. The sound source feature vector can be used to calculate the similarity between songs. In another example, a text feature vector v in which text information such as meta information of each song is expressed in a vector form can be used to calculate the similarity between songs together with a sound source feature vector have. The play list generating unit 240 may calculate the similarity between the sound source feature vector of the songs stored in the database and the sound source feature vector x0 of the seed song m0. For example, a cosine distance for calculating the distance between vectors may be used as the degree of similarity. The cosine distance (CD) between the two vectors a and b can be defined as: " (1) " That is, the smaller the distance between vectors is, the higher the similarity of the songs is, and the larger the distance between vectors is, the lower the similarity of the songs can be judged to be.

In addition to the cosine distance, the similarity between songs can be calculated using various methods such as Euclidean distance and Hamming distance. At this time, the sound source feature vector and the text feature vector may be jointly calculated or may be expressed as a sum of weights after calculating distances between vectors.

In step S402, the play list generating unit 240 may generate a separate candidate set M * by selecting a plurality of songs having similarities with the seed song m0 among the songs having unique characteristics stored in the database. For example, the play list generating unit 240 may select a song having similarity of a certain value or more among other songs called by the song of the seed song m0, or select a certain number of songs in descending order of similarity to generate a candidate set M * can do. As another example, the play list generator 240 may generate a candidate set M * by querying the database only for songs whose text feature vector distance between songs is less than a predetermined value.

In step S403, the play list generating unit 240 may select a song having a high degree of similarity with the currently reproduced song as a next reproduced candidate song based on the similarity between the songs whose unique characteristics are stored on the database. If the currently played song is mt, you can select the song with the highest similarity to the songs in the database as mt + 1, which will be played next time. At this time, the play list generating unit 240 may determine a song that has the greatest similarity to the currently reproduced song mt, or a randomly selected one of a plurality of songs selected on the basis of the similarity, as the next reproduction candidate song mt + 1. The already reproduced tune can be excluded from the selection target of the next reproduction candidate tune.

In step S404, the play list generator 240 calculates the similarity of at least one of the seed songs or the previous play songs to the candidate song selected in step S403, . At this time, the threshold value serves to prevent playback of a song that is too different from the seed song or the reference song while maintaining the diversity of the play list. In other words, the present invention may include logic for continuously checking the similarity between the seed song and the reference song through the threshold value in the next replay candidate song selection. For example, the threshold value can be arbitrarily selected from the distribution of the threshold values between all songs and can be set usually within 0.2 to 0.3. Thresholds can be used to balance or control the diversity and similarity of playlists. The reference song may be a song reproduced first, or a song having a history reproduced within a certain range (for example, within 5th, within 1 hour) after the first reproduction music.

In steps S405 and S406, if the similarity degree between the candidate song selected in step S403 and the seed song or the reference song is higher than the threshold value (i.e., the inter-vector distance is smaller than the critical distance in steps S405 and S406) , The candidate song selected in step S403 may be determined as the final next playable song and played back after the current playable song.

In steps S405 and S407, if the similarity degree between the candidate song selected in step S403 and the seed song or the reference song is lower than the threshold value (i.e., the inter-vector distance is larger than the critical distance in steps S405 and S407) It is possible to select any one piece of the songs included in the candidate set M * generated in the step S402 as the final next piece of music to reproduce after the current piece of music. For example, if the candidate song selected in step S403 is less than the threshold value of the seed song or the reference song, the play list generating unit 240 generates a play list A song can be selected and played back.

The playlist generating unit 240 repeats steps S403 to S407 in the above process, and can thereby infinitely generate a chain-shaped playlist based on the similarity between songs. Therefore, according to the present invention, diversity of the play list can be ensured by selecting the songs to be successively reproduced successively based on the similarity between songs, and at the same time, the similarity with the seed songs (or the reference songs) When the value is lower than a predetermined value, the similarity of the playlist can be maintained by replacing the song with a song having a high degree of similarity with the seed song (or the reference song).

As described above, according to the embodiments of the present invention, by using deep learning, eigenvalues of the sound source itself can be used as a factor for learning, so that the similarity between songs can be more accurately calculated. Furthermore, according to the embodiments of the present invention, it is possible to calculate the similarity between songs by using the meta information of the sound source as additional information as well as the characteristics of the sound source, and by using the meta information in the form of a filter, And by applying the logic to continuously check the similarity with the seed song, it is possible to maintain the diversity of the play list while at the same time preventing the inclusion of the seed song and other songs.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit, a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (3)

A method of generating playlists performed in a computer system,
Generating unique acoustic features of the sound source from the sound source data of the sound source content for each sound source content; And
Generating a playlist based on the similarity between the sound source contents by calculating a similarity degree between the sound source contents using the acoustic feature;
Lt; / RTI >
Wherein the generating the playlist comprises:
A play list similar to a current play song is selected based on the similarity between the sound source contents to generate a play list, and a similarity with a seed song is checked when a next play song is selected, Creating a playlist of the form
And generating a play list. A computer-readable recording medium having recorded thereon a program for causing the computer to execute the method of claim 1. A computer-implemented playlist generation system comprising:
An intrinsic feature processor for generating an intrinsic acoustic feature of the corresponding sound source from the sound source data of the sound source content for each sound source content; And
And generating a play list based on the similarity between the sound source contents by calculating a similarity degree between the sound source contents using the acoustic feature,
Lt; / RTI >
Wherein the playlist generating unit comprises:
A play list similar to the current play song is selected based on the similarity between the sound source contents to generate a play list, and a similarity to the seed song is checked when a next play song is selected, Creating lists
Wherein the playlist generation system comprises:

Images