KR102623419B1 - A method of collecting vocal data using a user's terminal and performing vocal evaluation using it - Google Patents

Abstract

A device according to an embodiment acquires the user's vocal data from the user's terminal, obtains song information based on the user's vocal data, and applies the user's vocal data and song information to a vocal evaluation artificial intelligence model to provide the user with a vocal evaluation. Output evaluation result data, calculate the user's vocal level based on the user's evaluation result data, generate user's vocalization properties based on the user's evaluation result data, and calculate the user's vocal level and user's vocalization. Properties are provided to the user's terminal.

Description

Vocal data collection using the user's terminal and vocal evaluation method using it { A METHOD OF COLLECTING VOCAL DATA USING A USER'S TERMINAL AND PERFORMING VOCAL EVALUATION USING IT }

The following embodiments relate to technology for collecting vocal data using a user's terminal and performing vocal evaluation using this.

Recently, with the development of broadcasting and communication technology, the scope of celebrities' activities has expanded and their daily lives have been exposed through broadcasting programs, leading to an increase in the number of aspiring celebrities. In order for an ordinary person to become a celebrity, they must put in effort, such as attending a specialized academy or participating in an agency's offline audition. To select aspiring celebrities, agencies cast ordinary people on the street or conduct periodic auditions to select aspiring celebrities. In both cases, a lot of time, money, and effort are required, such as when aspiring celebrities make efforts on their own or when agencies directly search for and cast aspiring celebrities.

Meanwhile, as online video service technology such as YouTube has developed, opportunities for aspiring celebrities to demonstrate their talents and skills have increased. Aspiring celebrities are trying to become celebrities by recording videos of themselves singing or dancing using mobile phones or camcorders and uploading them to YouTube to attract public attention. However, in these online auditions, it is difficult to accurately evaluate the participants' skills because they upload videos of themselves singing or dancing and then attract the public's interest simply by clicking on them.

In addition, there is a problem in that popularity rankings based on keyword searches provided by portal sites, etc. are not objective due to advertising functions such as Overture. In addition, even if a person is popular online, if he or she is cast by an agency, he or she must still undergo years of professional training and the agency must invest a lot of money and time.

In this environment, the Korean music industry developed rapidly following the Korean Wave, the so-called K-POP. Accordingly, the number of aspiring singers or idols has increased, and the market for these aspiring singers is also rapidly rising. However, despite the rapid increase in the number of aspiring celebrities, the stages (i.e., auditions or agencies) where aspiring celebrities can debut as celebrities are limited, and therefore, for a successful debut, it is important to find an agency that suits one's voice.

However, most of the aspirants had difficulty selecting the vocal training that was right for them because it was difficult to obtain information about each entertainment agency and there were no appropriate indicators regarding their vocalization or voice.

Therefore, there is a need for technology to objectively evaluate a user's vocals through the user's vocal data.

Korean Patent No. 10-1917216 (announced on November 9, 2018) Korean Patent No. 10-2107588 (announced on May 7, 2020) Korean Patent No. 10-2139889 (announced on July 30, 2020) Korean Patent No. 10-2259612 (announced on June 1, 2021)

Embodiments seek to perform a user's vocal evaluation based on the user's vocal data obtained from the user's terminal.

Embodiments seek to generate a user's vocal level and user's vocalization attributes through a vocal evaluation artificial intelligence model.

Embodiments seek to select vocal data to be evaluated for vocals among voice data acquired from a user's terminal.

According to one embodiment, a vocal data collection method using a user's terminal and a vocal evaluation method using the same include obtaining the user's vocal data from the user's terminal; Obtaining song information based on the user's vocal data; Applying the user's vocal data and the song information to a vocal evaluation artificial intelligence model to output the user's evaluation result data; calculating the user's vocal level based on the user's evaluation result data; generating vocalization attributes of the user based on the user's evaluation result data; and providing the user's vocal level and the user's vocalization attributes to the user's terminal.

The vocal evaluation artificial intelligence model includes an item evaluation model and a vocal attribute evaluation model, and the item evaluation model is a first distance, which is the distance between the sounds in the first set section, and a second, the height of the sound, through the song information. 1 Determine the height, determine the second distance, which is the distance between the sound of the second set section corresponding to the first set section, and the second height, which is the height of the sound, through the user's vocal data, and Compare the first distance and the second distance, and compare the first height and the second height so that the difference between the first distance and the second distance is within a preset first reference difference, and the first height If it is confirmed that the difference between the first height and the second height is within the preset second reference difference, O is output, and if the difference between the first distance and the second distance is not included within the first reference difference, or the first A pitch evaluation model that outputs A tempo evaluation model that generates a section that deviates from the standard BPM, outputs O if the section that deviates from the standard BPM is confirmed to be shorter than the preset standard time, and outputs , the number of times the vibrato is included is determined through the user's vocal data, and when it is confirmed that the vibrato satisfies the first standard number of times set in advance, O is output, and the vibrato does not satisfy the first standard number of times is output. A breathing control evaluation model that outputs If it is confirmed that the number of times that the corresponding bending and vibration is included satisfies the second preset standard number, O is output, and the number of times that the bending and vibration that is included corresponding to the reference rhythm is included satisfies the second standard number of times. A rhythm evaluation model that outputs A treble stability evaluation model that outputs O if it is confirmed that is smaller than the preset standard change, and outputs Generating the number of changes, generating the number of changes in the texture of the sound through the user's vocal data, generating the number of changes in volume size through the user's vocal data, the number of changes in the accent of the diction, and the number of changes in the texture of the sound If it is confirmed that the user's expression count, which is the sum of the number of times and the number of volume size changes, satisfies the preset third standard count, O is output, and if the user's expression count does not satisfy the third standard count, An expression evaluation model that outputs In this case, a completeness evaluation model that determines whether there is an impactful part through the user's vocal data, outputs O if it is confirmed that there is an impact part, and outputs an X if it is confirmed that there is no part with an impact. Includes, the speech attribute evaluation model checks the user's average voice frequency through the user's vocal data, and outputs H when it is confirmed that the user's average voice frequency is higher than a preset reference frequency, and the user's average voice frequency is confirmed to be higher than the preset reference frequency. A tone evaluation model that outputs L when it is confirmed that the average voice frequency is lower than or equal to the reference frequency, and confirms the user's average voice decibel through the user's vocal data, and determines the user's average voice decibel through the user's vocal data. Check whether the user's vocal cords are grounded, and if the user's average voice decibel is greater than a preset reference decibel and it is confirmed that the user's vocal cords are grounded, M is output, and the user's average voice decibel is greater than the reference decibel and the user's vocal cords are grounded. If it is confirmed that the user's vocal cords are grounded, output F, and if it is confirmed that the user's average voice decibel is less than or equal to the reference decibel and the user's vocal cords are grounded, output O, and if the user's average voice decibel is less than or equal to the reference decibel, output O. It includes an attribute evaluation model that outputs S if it is less than or equal to and it is confirmed that the user's vocal cords are not grounded.

Based on the user's evaluation result data, calculating the user's vocal level includes calculating the user's overall vocal level, and calculating the vocal level for each item, and calculating the user's overall vocal level. The step of calculating the level includes checking the total number of O output through the item evaluation model, and applying +1 to the total number of O to generate the user's comprehensive vocal level, The step of calculating the individual vocal level includes, when the user's comprehensive vocal level is generated, checking whether the output value of the item evaluation model corresponding to the confirmation item for which the vocal level is to be calculated is O or X. , if the output value of the item evaluation model corresponding to the confirmation item is confirmed to be O, assigning the vocal level of the confirmation item as a vocal level with the same level value as the user's comprehensive vocal level, corresponding to the confirmation item. If the output value of the item evaluation model is confirmed to be A step of assigning the vocal level of the level value generated by applying 3, and if it is confirmed that the confirmation item is not complete, the level value generated by applying -2 to the user's comprehensive vocal level as the vocal level of the confirmation item. A step of assigning a vocal level of, and generating a vocalization attribute of the user based on the evaluation result data of the user, includes a value output through the tone evaluation model and a value output through the attribute evaluation model. This is the step of combining values to create one of HF, LF, HM, LM, HS, LS, HO, and LO as the user's vocalization attribute.

Obtaining the user's vocal data from the user's terminal; when the user's voice is recognized through the user's terminal, the user's voice is not recognized for a preset set time from the time the user's voice is recognized. Acquiring voice data up to the point where the voice data is not heard, based on the voice data, identifying a pitch change whose pitch difference is greater than a preset target difference among the pitch changes included in the voice data, and determining a pitch change greater than the target difference. Generating the number of times as the number of confirmations, selecting voice data whose confirmation number is greater than a preset target number as candidate voice data, and extracting one of the candidate voice data and selecting it as the user's vocal data. And the step of extracting one of the candidate voice data and selecting it as the user's vocal data includes checking location information corresponding to the candidate voice data, and based on the location information, a location matching a preset effective location. Checking whether there is information; If it is confirmed that there is location information matching the effective location, selecting candidate voice data having location information matching the effective location as the user's vocal data; and When it is determined that there is no matching location information, randomly extracting one of the candidate voice data and selecting the extracted candidate voice data as the user's vocal data.

After calculating the user's vocal level based on the user's evaluation result data, it further includes generating the user's final vocal level by correcting the user's vocal level based on the evaluation obtained from the evaluator. And, based on the evaluation obtained from the evaluator, correcting the user's vocal level to generate the user's final vocal level includes uploading the user's vocal data to a vocal evaluation platform for a preset target period, Obtaining the evaluator's information and the evaluator's evaluation score from the evaluator's terminal who wishes to evaluate the user's vocal data through the vocal evaluation platform, based on the evaluator's information and the song information, the evaluator determining whether the evaluator is an original songwriter; if the evaluator is confirmed to be the original songwriter, applying a first weight to the evaluator's evaluation score; if it is confirmed that the evaluator is not the original songwriter, whether the evaluator has a job related to music Determining, if it is confirmed that the evaluator has an occupation related to music, applying a second weight smaller than the first weight to the evaluator's evaluation score, if it is confirmed that the evaluator has an occupation unrelated to music, Applying a third weight smaller than the second weight to the evaluator's evaluation score, and correcting the user's vocal level based on the weighted evaluation score of the evaluator.

A vocal data collection method using a user's terminal and a vocal evaluation method using the same include the steps of: checking location information corresponding to the user's vocal data and selecting an audition area based on the location information; Confirming that first content, which is content introducing the first audition to be held in the audition available area, has been produced and the first content has been uploaded to an audition recruitment site; Controlling the first content to be played and displayed on the user's terminal; When the second content, which is content uploaded by a first agency among agencies located within the audition available area, is uploaded to the audition recruitment site, the user's terminal is controlled to play and display the second content following the first content. steps; When third content, which is content uploaded by a second agency among agencies located within the audition available area, is uploaded to the audition recruitment site, the user's terminal is controlled to play and display the third content following the second content. steps; Controlling the user's terminal to play and display the first content following the third content; and controlling the second content and the third content to be played and displayed together following the first content in the user's terminal.

The device according to one embodiment may be combined with hardware and controlled by a computer program stored in a medium to execute any one of the above-described methods.

Embodiments may perform a user's vocal evaluation based on the user's vocal data obtained from the user's terminal.

Embodiments may generate the user's vocal level and the user's vocalization attributes through a vocal evaluation artificial intelligence model.

Embodiments may select vocal data to be evaluated for vocals among voice data acquired from the user's terminal.

Meanwhile, the effects according to the embodiments are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below.

1 is a diagram for explaining the configuration of a system according to an embodiment.
Figure 2 is a flowchart for explaining a process of performing vocal evaluation based on the user's vocal data according to an embodiment.
Figure 3 is a diagram for explaining a vocal evaluation artificial intelligence model according to an embodiment.
Figure 4 is a flowchart for explaining a process of calculating a user's vocal level based on the user's evaluation result data according to an embodiment.
Figure 5 is a flowchart for explaining the process of calculating the user's comprehensive vocal level according to one embodiment.
Figure 6 is a flowchart for explaining the process of calculating vocal levels for each item according to an embodiment.
Figure 7 is an example diagram of vocal levels and vocal grades generated based on output values of an evaluation model corresponding to each item including pitch, tempo, breathing, rhythm, high pitch, expressiveness, and completeness according to an embodiment.
Figure 8 is a flow chart to explain the process of generating user's vocalization attributes according to one embodiment.
Figure 9 is a diagram related to vocalization properties according to one embodiment.
FIG. 10 is a diagram illustrating an output interface of a user's terminal for providing the user's vocal level and user's vocalization properties to the user's terminal, according to an embodiment.
Figure 11 is a flowchart for explaining the process of selecting vocal data according to an embodiment.
Figure 12 is a flowchart for explaining the process of selecting vocal data from candidate voice data according to an embodiment.
Figure 13 is a flowchart for explaining a process for correcting a user's vocal level according to an embodiment.
FIG. 14 is a flowchart illustrating a process for providing audition content in progress in an audition available area to a user's terminal according to an embodiment.
Figure 15 is an exemplary diagram of the configuration of a device according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, various changes can be made to the embodiments, so the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents, or substitutes for the embodiments are included in the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

The terms used in the examples are for descriptive purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the embodiments belong. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

In addition, when describing with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiments, if it is determined that detailed descriptions of related known technologies may unnecessarily obscure the gist of the embodiments, the detailed descriptions are omitted.

Embodiments may be implemented in various types of products such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices.

An artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike existing rule-based smart systems, it is a system in which machines learn and make decisions on their own. As artificial intelligence systems are used, their recognition rates improve and they can more accurately understand user preferences, and existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning and element technologies using machine learning. Machine learning is an algorithmic technology that classifies/learns the characteristics of input data on its own, and elemental technology is a technology that mimics the functions of the human brain such as cognition and judgment by utilizing machine learning algorithms such as deep learning, including linguistic understanding and visual It consists of technical areas such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology that recognizes and applies/processes human language/characters and includes natural language processing, machine translation, conversation systems, question and answer, and voice recognition/synthesis. Visual understanding is a technology that recognizes and processes objects like human vision, and includes object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and image improvement. Inferential prediction is a technology that judges information to make logical inferences and predictions, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendations. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data creation/classification) and knowledge management (data utilization). Motion control is a technology that controls the autonomous driving of vehicles and the movement of robots, and includes motion control (navigation, collision, driving), operation control (behavior control), etc.

Generally, in order to apply machine learning algorithms to real life, learning is performed using a trial and error method due to the nature of the basic methodology of machine learning. In particular, deep learning requires hundreds of thousands of iterations. It is impossible to execute this in an actual physical external environment, so instead, the actual physical external environment is virtually implemented on a computer and learning is performed through simulation.

1 is a diagram for explaining the configuration of a system according to an embodiment.

Referring to FIG. 1, a system according to an embodiment may include a user terminal 100 and a device 200 that can communicate with each other through a communication network.

First, a communication network can be configured regardless of the communication mode, such as wired or wireless, and can be implemented in various forms to enable communication between servers and between servers and terminals.

The user terminal 100 is a terminal used by a user who wishes to receive vocal evaluation through vocal data, and may be implemented as a mobile phone, desktop PC, laptop PC, tablet PC, smartphone, etc., but is not limited to this. , It can also be implemented as various types of communication devices that can be connected to external servers. For example, as shown in FIG. 1, the user's terminal 100 may be a smartphone, and may be employed differently depending on the embodiment.

The user's terminal 100 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer. The user's terminal 100 may be configured to communicate with the device 200 wired or wirelessly.

The user's terminal 100 is connected to a website operated by a person or organization that provides services using the device 200, or an application developed and distributed by a person or organization that provides services using the device 200. The user's terminal 100 can be installed and linked with the device 200 through a website or application.

In FIG. 1 and the following description, for convenience of explanation, only the user terminal 100 is shown and described, but the number of terminals may vary depending on the embodiment. As long as the processing capacity of the device 200 allows, the number of terminals is not particularly limited.

That is, the singular expressions described in the claims may be understood to include the plural.

The device 200 may be its own server owned by a person or organization that provides services using the device 200, a cloud server, or a p2p (peer-to-peer) set of distributed nodes. It may be possible. The device 200 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer. The device 200 may be configured to communicate with the user's terminal 100 wired or wirelessly.

Additionally, the device 200 can communicate wired or wirelessly with websites including web pages including articles and SNS including blogs, cafes, Instagram, Facebook, Twitter, and YouTube, and the device 200 You can obtain information by accessing the website.

The device 200 obtains the user's vocal data from the user's terminal 100, obtains song information based on the user's vocal data, and applies the user's vocal data and song information to the vocal evaluation artificial intelligence model. Output the user's evaluation result data, calculate the user's vocal level based on the user's evaluation result data, generate the user's vocalization attribute based on the user's evaluation result data, and calculate the user's vocal level and the user's vocal level. The vocalization properties of can be provided to the user's terminal 100.

In the present invention, artificial intelligence (AI) refers to technology that imitates human learning ability, reasoning ability, perception ability, etc. and implements this with a computer, and includes concepts such as machine learning and symbolic logic. may include. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the results of the analysis, and makes judgments or predictions based on the results of the learning. Additionally, technologies that mimic the functions of the human brain, such as cognition and judgment, using machine learning algorithms can also be understood as the category of artificial intelligence. For example, technical fields such as verbal understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control may be included.

Machine learning can refer to the process of training a neural network model using experience processing data. Machine learning can mean that computer software improves its own data processing capabilities. A neural network model is built by modeling the correlation between data, and the correlation can be expressed by a plurality of parameters. A neural network model extracts and analyzes features from given data to derive correlations between data. Repeating this process to optimize the parameters of the neural network model can be called machine learning. For example, a neural network model can learn the mapping (correlation) between input and output for data given as input-output pairs. Alternatively, a neural network model may learn the relationships by deriving regularities between given data even when only input data is given.

An artificial intelligence learning model or neural network model may be designed to implement the human brain structure on a computer, and may include a plurality of network nodes with weights that simulate neurons of a human neural network. A plurality of network nodes may have a connection relationship with each other by simulating the synaptic activity of neurons in which neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes are located in layers of different depths and can exchange data according to convolutional connection relationships. The artificial intelligence learning model may be, for example, an artificial neural network model (Artificial Neural Network), a convolution neural network (CNN) model, etc. As an example, an artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, and Ada-boost. , Perceptron, Genetic Programming, Clustering, etc. can be used.

Among them, CNN is a type of multilayer perceptrons designed to use minimal preprocessing. CNN consists of one or several convolution layers and general artificial neural network layers on top of them, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize input data with a two-dimensional structure. Compared to other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained via standard back propagation. CNNs have the advantage of being easier to train and using fewer parameters than other feedforward artificial neural network techniques.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increasing size of available training data and the availability of computational power, combined with algorithmic advances such as piecewise linear unit and dropout training, have led to significant improvements in many computer vision tasks. For extremely large data sets, such as those available for many tasks today, overfitting is not critical, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. For this purpose, distributed, scalable implementations of deep neural networks can be used.

Figure 2 is a flowchart for explaining a process of performing vocal evaluation based on the user's vocal data according to an embodiment.

Referring to FIG. 2, first, in step S201, the device 200 may obtain the user's vocal data from the user's terminal 100.

Specifically, the device 200 may receive vocal data, which is data in which the user sang a song, from the user's terminal 100. At this time, the device 200 matches the user to the database provided in the device 200. User vocal data can be saved.

To this end, the user's terminal 100 may record video data or voice data in which the user sings a song, and generate the user's vocal data according to the recording results. At this time, the vocal data may be a video file or a voice file, but is not limited thereto.

The device 200 may collect voice, video, etc. directly from the user's terminal 100, or may obtain vocal data by collecting and processing data uploaded and registered to SNS services, etc. on the user's terminal 100.

Additionally, the device 200 may additionally obtain the user's personal information, which is information related to the user, from the user's terminal 100 in the process of acquiring the user's vocal data from the user's terminal 100. At this time, the user's personal information may include, but is not limited to, the user's name, user identification information including the user's nickname, and the user's gender, age, and physical characteristic information. The user's personal information may also include information about virtual people in the metaverse world.

The device 200 may provide a platform service or website for providing vocal evaluation to the user, and the user may upload vocal data to the platform service or website.

Additionally, the device 200 may store voices that can be used as vocal data based on voices recognized through the user's terminal 100 and select vocal data through the stored voices. For a detailed description in this regard, refer to FIG. 11.

In step S202, the device 200 may obtain song information based on the user's vocal data.

Specifically, the device 200 may be equipped with a song recognition artificial neural network or may communicate wired or wirelessly with a separate song recognition artificial neural network. When vocal data is input, the song recognition artificial neural network analyzes the voice included in the vocal data. It is an artificial neural network that identifies the melody of the voice and the lyrics of the voice, and outputs the original song information of the vocal data, that is, the song information of the vocal data, based on the melody of the voice and the lyrics of the voice. In addition, although not specifically written in relation to the song recognition artificial neural network, the song recognition artificial neural network can be implemented in the same form as the commonly used song search artificial neural network and music search artificial neural network.

That is, the device 200 can obtain song information corresponding to the user's vocal data by applying the user's vocal data to the song recognition artificial neural network. At this time, the song information may include the name of the song, the singer of the song, the audio file of the song, the score of the song, and the music genre of the song.

In step S203, the device 200 may apply the user's vocal data and song information to the vocal evaluation artificial intelligence model and output the user's evaluation result data.

Here, a vocal evaluation artificial intelligence model may be provided in the device 200, and the vocal evaluation artificial intelligence model may include an item evaluation model and a vocal attribute evaluation model.

Here, the item evaluation model is a model that evaluates items, and the items may include pitch, tempo, breathing, rhythm, high pitch, expressiveness, and completeness. In other words, the item evaluation model includes a pitch evaluation model that evaluates the sense of pitch among the items, a tempo sense evaluation model that evaluates the sense of tempo among the items, a breathing control evaluation model that evaluates breathing among the items, and a model that evaluates the sense of rhythm among the items. It may include a rhythm evaluation model, a high-pitched stability evaluation model that is a model that evaluates high notes among items, an expressiveness evaluation model that is a model that evaluates expressiveness among items, and a completeness evaluation model that is a model that evaluates completeness among items.

Additionally, the speech attribute evaluation model may include a tone evaluation model that evaluates the user's tone, and an attribute evaluation model that evaluates the user's attributes.

Refer to FIG. 3 for a detailed explanation regarding the vocal evaluation artificial intelligence model.

In step S204, the device 200 may calculate the user's vocal level based on the user's evaluation result data.

Specifically, the device 200 may calculate the user's vocal level based on the user's evaluation result data output by applying the user's vocal data and song information to the vocal evaluation artificial intelligence model. For detailed description in this regard, refer to FIGS. 4 to 7.

In step S205, the device 200 may generate the user's vocalization attributes based on the user's evaluation result data.

Specifically, the device 200 may apply the user's vocal data and song information to a vocal evaluation artificial intelligence model and generate the user's vocalization properties based on the output user evaluation result data. For a detailed description in this regard, refer to FIGS. 8 and 9.

In step S206, the device 200 may provide the user's vocal level and the user's vocalization properties to the user's terminal 100.

Specifically, the device 200 applies the user's vocal data and song information to the vocal evaluation artificial intelligence model and calculates the user's vocal level and the user's vocal data and song information based on the user's evaluation result data output. Based on the user's evaluation result data output by applying it to the vocal evaluation artificial intelligence model, the user's vocalization properties generated can be provided to the user's terminal 100. At this time, refer to FIG. 10 for an example in which the user's vocal level and the user's vocalization properties are provided to the user's terminal 100.

That is, the device 200 provides the user's vocal level and the user's vocalization properties to the user, so that the user can objectively determine the current user's vocal skills based on the user's vocal level and the user's vocalization properties. .

In addition, the user can find out vocal training suitable for the user based on the user's vocal level and the user's vocalization properties, thereby effectively improving vocal skills.

Figure 3 is a diagram for explaining a vocal evaluation artificial intelligence model according to an embodiment.

Specifically, the vocal evaluation artificial intelligence model may include an item evaluation model and a vocal attribute evaluation model.

In addition, the item evaluation models include the pitch evaluation model, which is a model that evaluates the sense of pitch among items, the tempo sense evaluation model, which is a model that evaluates the sense of tempo among items, the breathing control evaluation model, which is a model that evaluates breathing among items, and the model that evaluates rhythm among items. It may include a rhythm evaluation model, a high-pitched stability evaluation model that is a model that evaluates high notes among items, an expressiveness evaluation model that is a model that evaluates expressiveness among items, and a completeness evaluation model that is a model that evaluates completeness among items.

The pitch evaluation model is a model that evaluates whether the user expressed the sound accurately by comparing the pitch included in the user's vocal data with the pitch included in the song information. The pitch evaluation model is a first setting section preset through song information. The first distance, which is the distance between the notes, and the first height, which is the height of the sound, are identified, and the second distance, which is the distance between the notes in the second set section corresponding to the first set section, is determined through the user's vocal data. Determine the distance and the second height, which is the height of the sound, compare the first distance and the second distance, compare the first height and the second height, and the difference between the first distance and the second distance is a preset first reference difference. If it is confirmed that the difference between the first height and the second height is within the preset second reference difference, O is output as an output value, and the difference between the first distance and the second distance is not included within the first reference difference. Alternatively, if it is confirmed that the difference between the first height and the second height is not within the second reference difference, it may be a model that outputs X as an output value. Additionally, the pitch evaluation model may be an artificial intelligence model, and the pitch evaluation model may be implemented in the form of a convolutional neural network (CNN).

Specifically, the device 200 may preprocess the audio file of the song included in the song information and the user's vocal data to evaluate the user's pitch through a pitch evaluation model. At this time, the device 200 considers the possibility that the length of the song's audio file and the vocal data are different, and processes the song's audio file and the user's vocal data by cropping them to a certain length (5 to 10 seconds). can be preprocessed, and the device 200 can also perform preprocessing to remove noise from the audio file of the song and the user's vocal data. Furthermore, the device 200 may efficiently extract information from voice through a spectrum and preprocess the audio file of the song and the user's vocal data. At this time, the process for performing preprocessing has not been specifically written, but preprocessing can be performed by a commonly used method. Additionally, the device 200 may obtain an output value by inputting the audio file of the song and the user's vocal data included in the preprocessed song information into the pitch evaluation model. At this time, the pitch evaluation model generates the distance between the notes through the pre-processed audio file of the song, creates the distance between the notes through the pre-processed user's vocal data, and creates the difference between the two distances. , the pitch is generated from the pre-processed audio file of the song, the pitch is generated from the pre-processed user's vocal data to create the difference between the two heights, and the difference between the two distances and the difference between the two heights are both calculated in advance. If it is confirmed that the set standard is satisfied, O can be output as the output value, and if it is confirmed that at least one of the difference between the two distances and the difference between the two heights does not satisfy the standard, X can be output as the output value.

The tempo feeling evaluation model is a model that evaluates whether the user has the sense to keep the BPM constant by comparing the tempo included in the user's vocal data with the tempo included in the song information. The tempo feeling evaluation model evaluates the standard BPM through song information. Identify the section that deviates from the standard BPM through the user's vocal data, and if it is confirmed that the section that deviates from the standard BPM among the total sections included in the user's vocal data is shorter than the preset standard time, O is output as the output value. , If it is confirmed that the section outside the standard BPM among the total sections included in the user's vocal data is not shorter than the standard time, it may be a model that outputs X as the output value. Additionally, the tempo sense evaluation model may be an artificial intelligence model, and the tempo sense evaluation model may be implemented in the form of a convolutional neural network (CNN).

Specifically, the device 200 may preprocess the audio file of the song included in the song information and the user's vocal data to evaluate the user's sense of tempo through a tempo sense evaluation model. At this time, the device 200 considers the possibility that the length of the song's audio file and the vocal data are different, and processes the song's audio file and the user's vocal data by cropping them to a certain length (5 to 10 seconds). can be preprocessed, and the device 200 can also perform preprocessing to remove noise from the audio file of the song and the user's vocal data. Furthermore, the device 200 may efficiently extract information from voice through a spectrum and preprocess the audio file of the song and the user's vocal data. At this time, the process for performing preprocessing has not been specifically written, but preprocessing can be performed by a commonly used method. Additionally, the device 200 may obtain an output value by inputting the audio file of the song and the user's vocal data included in the preprocessed song information into the tempo evaluation model. At this time, the tempo evaluation model generates the standard BPM, which is the BPM of the song, through the pre-processed song information, and checks the sections that deviate from the standard BPM through the pre-processed user's vocal data to determine the total section included in the user's vocal data. If it is confirmed that the section outside the standard BPM is short, O can be output as the output value, and if it is confirmed that the section outside the standard BPM among the total sections included in the user's vocal data is not short, X can be output as the output value.

The breathing control evaluation model is a model that evaluates whether the user can control breathing by checking the number of times vibrato included in the user's vocal data. The breathing control evaluation model evaluates whether the user can control breathing by checking the number of times vibrato included in the user's vocal data. By checking the number of times, if it is confirmed that the number of times the vibrato is included in the user's vocal data satisfies the first preset standard number, O is output as an output value, and the number of times the vibrato is included in the user's vocal data is the first standard number. If it is determined that is not satisfied, it may be a model that outputs X as the output value. Additionally, the respiratory control evaluation model may be an artificial intelligence model, and the respiratory control evaluation model may be implemented in the form of a convolutional neural network (CNN). Here, vibrato is a technique that expresses continuous vibration that gradually goes up and down the pitch of the sound, and can be realized by changing the frequency of the sound up and down within a preset range. Here, the first reference number of times may be a preset number of times. In this case, the preset number of times may be one number of times or a range of times. Additionally, the first reference number may be corrected as the number of embodiments through the respiratory control evaluation model increases. That is, if the number of times the vibrato is included in the user's vocal data is within the first standard number of times, it can be confirmed that the first standard number of times is satisfied, and the number of times the user's vocal data includes the vibrato is not within the first standard number of times. If not, it can be confirmed that the first standard number of times is not satisfied.

Specifically, the device 200 may preprocess the user's vocal data to evaluate the user's breathing through a breathing control evaluation model. At this time, the device 200 may perform preprocessing to remove noise from the user's vocal data. Furthermore, the device 200 may efficiently extract information from voice through a spectrum and preprocess the user's vocal data. At this time, the process for performing preprocessing has not been specifically written, but preprocessing can be performed by a commonly used method. Additionally, the device 200 may obtain an output value by inputting pre-processed user vocal data into a respiratory control evaluation model. At this time, the breathing control evaluation model generates the number of times vibrato is included through the preprocessed user's vocal data, and when it is confirmed that the generated number of times including vibrato satisfies the first standard number, it outputs O as the output value. , if it is confirmed that the number of times the vibrato is included does not satisfy the first standard number of times, X can be output as the output value.

The rhythm evaluation model is a model that evaluates whether the user has a sense of rhythm by identifying the rhythm based on song information and checking the number of times the user's vocal data includes bending and vibration corresponding to the rhythm. Identify the reference rhythm through the information, check the number of times bending and vibration corresponding to the reference rhythm are included through the user's vocal data, and determine in advance the number of times the user's vocal data includes bending and vibration corresponding to the reference rhythm. If it is confirmed that more than the set second standard number is included, O is output as the output value, and if it is confirmed that the user's vocal data contains bending and vibration corresponding to the standard rhythm less than the second standard number, the output value is X. It may be a model that outputs. Additionally, the rhythm evaluation model may be an artificial intelligence model, and the rhythm evaluation model may be implemented in the form of a convolutional neural network (CNN). Here, bending is to adjust the pitch of the sound, and the way to check the bending corresponding to the standard rhythm is to check whether the bending is suitable for the standard rhythm by checking the voice during the connection process made during the connection time between the bending start sound and the sound to be reached. You can check, and vibration represents the tremor of the sound, similar to vibrato, but expresses regular vibration and an even frequency of change in pitch of the sound. The method of checking the vibration corresponding to the standard rhythm is to check the vibration corresponding to the standard rhythm. You can check whether there has been a change in pitch of the sound. Here, the second reference number of times may be a preset number of times. In this case, the preset number of times may be one number of times or a range of times. Additionally, the second reference number may be corrected as the number of examples through the rhythm evaluation model increases. That is, if the number of bending and vibrations corresponding to the standard rhythm is included in the user's vocal data within the second standard number, it can be confirmed that the second standard number is satisfied, and the bending and vibration corresponding to the standard rhythm are included in the user's vocal data. And when the number of times the vibration is included is not included in the second standard number of times, it can be confirmed that the second standard number of times is not satisfied.

Specifically, the device 200 may preprocess the audio file of the song included in the song information and the user's vocal data to evaluate the user's sense of rhythm through a rhythm evaluation model. At this time, the device 200 considers the possibility that the length of the song's audio file and the vocal data are different, and processes the song's audio file and the user's vocal data by cropping them to a certain length (5 to 10 seconds). can be preprocessed, and the device 200 can also perform preprocessing to remove noise from the audio file of the song and the user's vocal data. Furthermore, the device 200 may efficiently extract information from voice through a spectrum and preprocess the audio file of the song and the user's vocal data. At this time, the process for performing preprocessing has not been specifically written, but preprocessing can be performed by a commonly used method. Additionally, the device 200 may obtain an output value by inputting pre-processed user vocal data into a rhythm evaluation model. At this time, the rhythm evaluation model generates the number of times that bending and vibrations corresponding to the reference rhythm are included through the preprocessed user's vocal data, and the number of times that bending and vibrations corresponding to the generated reference rhythm are included is the second reference number. If it is confirmed that it satisfies, O can be output as an output value, and if it is confirmed that the number of bending and vibrations included corresponding to the standard rhythm does not satisfy the second standard number, X can be output as an output value.

The treble stability evaluation model is a model that evaluates whether the user's high notes are stable within the user's vocal range by checking the user's vocal range included in the user's vocal data. The treble stability evaluation model evaluates the user's vocal range through the user's vocal data. Confirm, generate a standard treble height based on the user's vocal range, check the change in sound from the standard treble height through the user's vocal data, and output if it is confirmed that the change in sound at the standard treble height is less than the preset standard change. It may be a model that outputs O as a value, and outputs X as an output value if it is confirmed that the change in sound from the standard treble height is not small. Additionally, the treble stability evaluation model may be an artificial intelligence model, and the treble stability evaluation model may be implemented in the form of a convolutional neural network (CNN).

Specifically, the device 200 may preprocess the user's vocal data to evaluate the user's high pitch through a high pitch stability evaluation model. At this time, the device 200 may perform preprocessing to remove noise from the user's vocal data. Furthermore, the device 200 may efficiently extract information from voice through a spectrum and preprocess the user's vocal data. At this time, the process for performing preprocessing has not been specifically written, but preprocessing can be performed by a commonly used method. Additionally, the device 200 may obtain an output value by inputting the preprocessed user's vocal data into the treble stability evaluation model. At this time, the treble stability evaluation model checks the user's vocal range through the preprocessed user's vocal data, generates a reference treble height based on the user's vocal range, checks the change in sound at the standard treble height, and determines the reference treble height. If it is confirmed that the sound change is smaller than the standard change, O can be output as the output value, and if it is confirmed that the sound change in the standard treble height is not smaller than the standard change, X can be output as the output value.

The expressiveness evaluation model is a model that evaluates whether the user can express himself by checking the change in diction stress, sound texture, and volume size included in the user's vocal data. Check the number of times the accent of the diction has changed, check the number of times the texture of the sound has changed through the user's vocal data, and check the number of times the volume has changed through the user's vocal data to determine the number of times the accent of the diction has changed, If it is confirmed that the number of sound texture changes and the number of volume size changes satisfy the preset third standard number, O is output as the output value, and the number of diction stress changes, the number of sound texture changes, and the number of volume size changes are set. 3 If it is confirmed that the standard number of times is not met, the model may output X as the output value. Additionally, the expression evaluation model may be an artificial intelligence model, and the expression evaluation model may be implemented in the form of a convolutional neural network (CNN). Here, the third reference number of times may be a preset number of times. In this case, the preset number of times may be one number of times or a range of times. Additionally, the third reference number may be corrected as the number of embodiments through the expressiveness evaluation model increases. In other words, if the number of times the accent of the diction included in the user's vocal data has changed, the number of times the texture of the sound has changed, and the number of times the volume has changed are within the third standard number of times, it can be confirmed that the third standard number of times is satisfied, and the user If the number of times the accent of diction, the number of times the texture of the sound has changed, and the number of times the volume has changed included in the vocal data are not included in the third standard number of times, it can be confirmed that the third standard number of times is not satisfied.

Specifically, the device 200 may preprocess the user's vocal data to evaluate the user's expressiveness through an expressiveness evaluation model. At this time, the device 200 may perform preprocessing to remove noise from the user's vocal data. Furthermore, the device 200 may efficiently extract information from voice through a spectrum and preprocess the user's vocal data. At this time, the process for performing preprocessing has not been specifically written, but preprocessing can be performed by a commonly used method. Additionally, the device 200 may obtain an output value by inputting pre-processed user vocal data into an expressiveness evaluation model. At this time, the expressiveness evaluation model checks the number of times the diction stress has changed through the pre-processed user's vocal data, checks the number of times the sound texture has changed through the pre-processed user's vocal data, and checks the number of times the sound texture has changed through the pre-processed user's vocal data. Check the number of times the volume size has changed through the data, and if it is confirmed that the number of times the stress of the confirmed diction has changed, the number of times the sound texture has changed, and the number of times the volume size has changed satisfies the third standard number of times, the output value is O. is output, and if it is confirmed that the number of times the accent of the diction has changed, the number of times the sound texture has changed, and the number of times the volume has changed do not meet the third standard number of times, X can be output as the output value.

The completeness evaluation model is a part that has impact through the user's vocal data when O is output as an output value in all of the pitch evaluation model, tempo evaluation model, breathing control evaluation model, rhythm evaluation model, treble stability evaluation model, and expressiveness evaluation model. It is a model that evaluates whether there is a completeness evaluation model, and the completeness evaluation model is an output value of When output, the completeness evaluation model may output X as the output value without analyzing the user's vocal data. In other words, it is possible to communicate wired or wirelessly with the completion evaluation model, tempo evaluation model, breathing control evaluation model, rhythm evaluation model, treble stability evaluation model, and expressiveness evaluation model. The completeness evaluation model checks whether there is an impactful part through the user's vocal data. If it is confirmed that there is an impactful part in the user's vocal data, it outputs O as an output value, and determines whether there is an impactful part in the user's vocal data. If it is confirmed that there is no part, it may be a model that outputs X as the output value. Additionally, the completeness evaluation model may be an artificial intelligence model, and the completeness evaluation model may be implemented in the form of a convolutional neural network (CNN).

Specifically, the device 200 may preprocess the user's vocal data to evaluate the user's completeness through a completeness evaluation model. At this time, the device 200 may perform preprocessing to remove noise from the user's vocal data. Furthermore, the device 200 may efficiently extract information from voice through a spectrum and preprocess the user's vocal data. At this time, the process for performing preprocessing has not been specifically written, but preprocessing can be performed by a commonly used method. Additionally, the device 200 may obtain an output value by inputting pre-processed user vocal data into a completeness evaluation model. At this time, the completeness evaluation model outputs O as an output value if it is confirmed that there is an impactful part through the pre-processed user's vocal data, and if it is confirmed that there is no impactful part through the pre-processed user's vocal data, it is output. You can output X as the value.

Additionally, the speech attribute evaluation model may include a tone evaluation model that evaluates the user's tone, and an attribute evaluation model that evaluates the user's attributes.

The tone evaluation model checks the user's average voice frequency through the user's vocal data, and outputs H as an output value if it is confirmed that the user's average voice frequency is higher than the preset reference frequency. If it is confirmed that it is not high, it may be a model that outputs L as the output value. Additionally, the tone evaluation model may be an artificial intelligence model, and the tone evaluation model may be implemented in the form of a convolutional neural network (CNN). Here, the reference frequency is a preset frequency value and may vary depending on the embodiment, and the reference frequency may be corrected as the number of embodiments through the tone evaluation model increases.

Specifically, the device 200 may preprocess the user's vocal data to evaluate the user's tone through a tone evaluation model. At this time, the device 200 may perform preprocessing to remove noise from the user's vocal data. Furthermore, the device 200 may efficiently extract information from voice through a spectrum and preprocess the user's vocal data. At this time, the process for performing preprocessing has not been specifically written, but preprocessing can be performed by a commonly used method. Additionally, the device 200 may obtain an output value by inputting the pre-processed user's vocal data into the tone evaluation model. At this time, the tone evaluation model checks the user's average voice frequency through the preprocessed user's vocal data, and if it is confirmed that the user's average voice frequency is higher than the preset reference frequency, it outputs H as an output value, and the user's average voice frequency is confirmed to be higher than the preset reference frequency. If it is confirmed that the frequency is not higher than the reference frequency, L can be output as the output value.

The attribute evaluation model checks the user's average voice decibel through the user's vocal data, and checks whether the user's vocal cords are grounded through the user's vocal data, so that the user's average voice decibel is greater than the preset reference decibel and the user's vocal cords are grounded. If it is confirmed that the user's average voice decibel is greater than the standard decibel and the user's vocal cords are not grounded, output F as the output value, and if the user's average voice decibel is greater than the standard decibel and the user's vocal cords are not grounded, output F as the output value. If it is confirmed that the user's vocal cords are grounded, O is output as an output value, and if it is confirmed that the user's average voice decibel is not greater than the reference decibel and the user's vocal cords are not grounded, it may be a model that outputs S as an output value. Additionally, the attribute evaluation model may be an artificial intelligence model, and the attribute evaluation model may be implemented in the form of a convolutional neural network (CNN). Here, the reference decibel is a preset decibel value and may vary depending on the embodiment, and the reference decibel may be corrected as the number of embodiments through the attribute evaluation model increases.

Specifically, the device 200 may preprocess the user's vocal data to evaluate the user's attributes through an attribute evaluation model. At this time, the device 200 may perform preprocessing to remove noise from the user's vocal data. Furthermore, the device 200 may efficiently extract information from voice through a spectrum and preprocess the user's vocal data. At this time, the process for performing preprocessing has not been specifically written, but preprocessing can be performed by a commonly used method. Additionally, the device 200 may obtain an output value by inputting pre-processed user vocal data into an attribute evaluation model. At this time, the attribute evaluation model checks the user's average voice decibel through the preprocessed user's vocal data, and checks whether the user's vocal cords are grounded through the user's vocal data, so that the user's average voice decibel is higher than the preset reference decibel. If it is confirmed that the user's vocal cords are grounded, M is output as the output value. If the user's average voice decibel is greater than the reference decibel and it is confirmed that the user's vocal cords are not grounded, F is output as the output value. If it is confirmed that the user's vocal cords are not grounded and not greater than the standard decibel, O is output as the output value. If it is confirmed that the user's average voice decibel is not greater than the standard decibel and the user's vocal cords are not grounded, S can be output as the output value. .

Meanwhile, the pitch evaluation model, tempo evaluation model, breathing control evaluation model, rhythm evaluation model, treble stability evaluation model, expressiveness evaluation model, completeness evaluation model, tone evaluation model, and attribute evaluation model can communicate with each other wired or wirelessly.

Meanwhile, the device 200 stores the vocal data of the user for whom the vocal evaluation has been completed in an artificial intelligence learning-based evaluation model database, and includes a pitch evaluation model, a tempo evaluation model, a breathing control evaluation model, a rhythm evaluation model, a treble stability evaluation model, and expressiveness. It can be used as learning data, which is data for training an evaluation model, completeness evaluation model, tone evaluation model, and attribute evaluation model.

The device 200 can preprocess learning data, and the device 200 can use the preprocessed learning data to expand to various data, such as data with accompaniment, data without accompaniment, and data with noise. AUGMENTATION processing, etc. may be exemplified. For AUGMENTATION processing, deep learning techniques can be used, traditional methods such as RoomImpulse Response can be used, and further information can be efficiently extracted from speech through a spectrum.

In addition, the artificial intelligence learning-based evaluation model database can build a learning database by applying vocal data transformed into a spectrum to a convolutional neural network (CNN) commonly used in images. To this end, the artificial intelligence learning-based evaluation model database can achieve the highest efficiency even with small data by using pre-trained weights specialized for vocal data.

Furthermore, the artificial intelligence learning-based evaluation model database can use softmax and Cross Entropy Loss as classification models to effectively process vocal evaluation, and Mean Squared Loss can be used for learning to determine intuition level and item level. A back propagation algorithm can be used.

The artificial intelligence learning-based evaluation model database according to an embodiment of the present invention preserves privacy by building a data flow pipeline and can continuously update the performance of the model through continuous learning, and can be used universally for various services. An applicable API may be configured. Accordingly, an evaluation process using an artificial intelligence learning-based evaluation model database can be provided through various services such as mobile phone application services and metaverse services.

Figure 4 is a flowchart for explaining a process of calculating a user's vocal level based on the user's evaluation result data according to an embodiment.

Referring to FIG. 4, first, in step S401, the device 200 may calculate the user's comprehensive vocal level based on the output values output by the all item evaluation model.

Specifically, the device 200 is an output value output through a pitch evaluation model, an output value output through a tempo evaluation model, an output value output through a breathing control evaluation model, an output value output through a rhythm evaluation model, and a high pitch. The user's comprehensive vocal level can be calculated based on the output value output through the stability evaluation model, the output value output through the expressiveness evaluation model, and the output value output through the completeness evaluation model. Refer to FIG. 5 for a detailed description regarding the process of calculating the user's comprehensive vocal level.

In step S402, the device 200 may calculate the vocal level for each item based on the output value output by the item evaluation model corresponding to the item for which the vocal level is to be calculated.

Specifically, the device 200 may calculate the level corresponding to each item among items including pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, as well as the user's overall vocal level. Refer to FIG. 6 for a detailed explanation regarding the process of calculating the vocal level for each item.

That is, the device 200 can calculate the user's overall vocal level and the vocal level for each item and provide them to the user.

Figure 5 is a flowchart for explaining the process of calculating the user's comprehensive vocal level according to one embodiment.

Referring to FIG. 5, first, in step S501, the device 200 can check the total number of O output through the item evaluation model.

Specifically, the device 200 is an output value output through a pitch evaluation model, an output value output through a tempo evaluation model, an output value output through a breathing control evaluation model, an output value output through a rhythm evaluation model, and a high pitch. Based on the output value output through the stability evaluation model, the output value output through the expressiveness evaluation model, and the output value output through the completeness evaluation model, the total number of O among the output values output through the item evaluation model can be checked. there is.

For example, as a result of applying the first user's vocal data to the vocal evaluation artificial intelligence model, the output value output through the pitch evaluation model is O, the output value output through the tempo evaluation model is O, and the breathing control evaluation model The output value output through is O, the output value output through the rhythm evaluation model is O, the output value output through the treble stability evaluation model is X, and the output value output through the expressiveness evaluation model is X, If the output value output through the completeness evaluation model is , Confirm that the output value output through the breathing control evaluation model is O, confirm that the output value output through the rhythm evaluation model is O, and confirm that the output value output through the treble stability evaluation model is X. , by confirming that the output value output through the expressiveness evaluation model is X, and by confirming that the output value output through the completeness evaluation model is

In step S502, the device 200 may generate the user's overall vocal level by applying +1 to the total number of O's.

Specifically, the device 200 is an output value output through a pitch evaluation model, an output value output through a tempo evaluation model, an output value output through a breathing control evaluation model, an output value output through a rhythm evaluation model, and a high pitch. By checking the output value output through the stability evaluation model, the output value output through the expressiveness evaluation model, and the output value output through the completeness evaluation model, you can check the total number of O output through the item evaluation model, and confirm The user's overall vocal level can be calculated by applying +1 to the total number of O's output through the item evaluation model.

For example, as a result of applying the first user's vocal data to the vocal evaluation artificial intelligence model, the output value output through the pitch evaluation model is O, the output value output through the tempo evaluation model is O, and the breathing control evaluation model The output value output through is O, the output value output through the rhythm evaluation model is O, the output value output through the treble stability evaluation model is X, and the output value output through the expressiveness evaluation model is X, If the output value output through the completeness evaluation model is , Confirm that the output value output through the breathing control evaluation model is O, confirm that the output value output through the rhythm evaluation model is O, and confirm that the output value output through the treble stability evaluation model is X. , it is confirmed that the output value output through the expressiveness evaluation model is X, and the output value output through the completeness evaluation model is confirmed to be By applying +1 to 4, the total number of O output through the item evaluation model, the user's overall vocal level can be created as 5.

Figure 6 is a flowchart for explaining the process of calculating vocal levels for each item according to an embodiment.

Referring to FIG. 6, first, in step S601, the device 200 may check whether the output value of the item evaluation model corresponding to the confirmation item is O.

Specifically, when the user's comprehensive vocal level is generated, the device 200 selects the confirmation item for which the vocal level is to be calculated among pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, and includes it in the item evaluation model. You can check whether it is O or X by checking the output value of the item evaluation model corresponding to the confirmed item among the evaluation models. At this time, the device 200 can calculate vocal levels corresponding to all items.

For example, as a result of applying the first user's vocal data to the vocal evaluation artificial intelligence model, the output value output through the pitch evaluation model is O, the output value output through the tempo evaluation model is O, and the breathing control evaluation model The output value output through is O, the output value output through the rhythm evaluation model is O, the output value output through the treble stability evaluation model is X, and the output value output through the expressiveness evaluation model is X, If the output value output through the completeness evaluation model is It can be confirmed that the output value output through the tempo feeling evaluation model corresponding to the tempo feeling is O.

If the output value of the item evaluation model corresponding to the confirmation item is confirmed to be O in step S601, in step S602, the device 200 assigns the vocal level of the confirmation item as a vocal level with the same level value as the user's comprehensive vocal level. can do.

Specifically, when the user's comprehensive vocal level is generated, the device 200 selects the confirmation item for which the vocal level is to be calculated among pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, and includes it in the item evaluation model. Among the confirmed evaluation models, the output value of the item evaluation model corresponding to the confirmation item is checked, and if the output value of the item evaluation model corresponding to the confirmation item is confirmed to be O, the vocal level of the confirmation item is set to the same level as the user's comprehensive vocal level. The value can be given as a vocal level.

For example, as a result of applying the first user's vocal data to the vocal evaluation artificial intelligence model, the output value output through the pitch evaluation model is O, the output value output through the tempo evaluation model is O, and the breathing control evaluation model The output value output through is O, the output value output through the rhythm evaluation model is O, the output value output through the treble stability evaluation model is X, and the output value output through the expressiveness evaluation model is X, If the output value output through the completeness evaluation model is Through the tempo feeling evaluation model corresponding to the tempo feeling item, it can be confirmed that the output value is O, and the vocal level of the tempo feeling can be assigned a vocal level of 5, which is the same level value as the user's comprehensive vocal level of 5. .

If it is confirmed that the output value of the item evaluation model corresponding to the confirmation item is X in step S601, the device 200 may determine whether the confirmation item is complete in step S603.

Specifically, when the user's comprehensive vocal level is generated, the device 200 selects the confirmation item for which the vocal level is to be calculated among pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, and includes it in the item evaluation model. Among the evaluation models, if the output value of the item evaluation model corresponding to the confirmation item is checked and the output value of the item evaluation model corresponding to the confirmation item is confirmed to be X, it can be determined whether the confirmation item is complete.

If the confirmation item is confirmed to be complete in step S603, the device 200 may assign the vocal level of the confirmation item to a vocal level of a level value generated by applying -3 to the user's overall vocal level in step S604. At this time, the lowest value of the level can be unconditionally created as 1.

Specifically, when the user's comprehensive vocal level is generated, the device 200 selects the confirmation item for which the vocal level is to be calculated among pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, and includes it in the item evaluation model. Among the evaluated evaluation models, the output value of the item evaluation model corresponding to the confirmation item is checked, and if the output value of the item evaluation model corresponding to the confirmation item is confirmed to be If it is determined that this is the case, the vocal level of the confirmation item can be assigned the vocal level of the level value generated by applying -3 to the user's overall vocal level.

For example, as a result of applying the first user's vocal data to the vocal evaluation artificial intelligence model, the output value output through the pitch evaluation model is O, the output value output through the tempo evaluation model is O, and the breathing control evaluation model The output value output through is O, the output value output through the rhythm evaluation model is O, the output value output through the treble stability evaluation model is X, and the output value output through the expressiveness evaluation model is X, If the output value output through the completeness evaluation model is Through the completeness evaluation model corresponding to the completeness level, which is an item to be checked, it can be confirmed that the output value is Thus, the vocal level of the generated level value can be assigned as 2.

Meanwhile, the lowest value of the vocal level of the confirmation item can be unconditionally created as 1. For example, if the user's overall vocal level is 2, the check item is completeness, and the output value output through the completeness evaluation model is However, since the lowest value of the vocal level is unconditionally 1, the device 200 can generate a complete vocal level of 1. .

If it is determined in step S603 that the confirmation item is not complete, in step S605, the device 200 may assign the vocal level of the confirmation item as a vocal level of a level value generated by applying -2 to the user's overall vocal level. . At this time, the lowest value of the level can be unconditionally created as 1.

Specifically, when the user's comprehensive vocal level is generated, the device 200 selects the confirmation item for which the vocal level is to be calculated among pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, and includes it in the item evaluation model. Among the evaluated evaluation models, the output value of the item evaluation model corresponding to the confirmation item is checked, and if the output value of the item evaluation model corresponding to the confirmation item is confirmed to be If it is determined that it is not, the vocal level of the confirmation item can be given as the vocal level of the level value generated by applying -2 to the user's overall vocal level.

For example, as a result of applying the first user's vocal data to the vocal evaluation artificial intelligence model, the output value output through the pitch evaluation model is O, the output value output through the tempo evaluation model is O, and the breathing control evaluation model The output value output through is O, the output value output through the rhythm evaluation model is O, the output value output through the treble stability evaluation model is X, and the output value output through the expressiveness evaluation model is X, If the output value output through the completeness evaluation model is It can be confirmed that the output value output is By applying -2, the vocal level of the generated level value can be given as 3.

Meanwhile, the lowest value of the vocal level of the confirmation item can be unconditionally created as 1. For example, if the user's overall vocal level is 2, the check item is expressiveness, and the output value output through the expressiveness evaluation model is However, since the lowest value of the vocal level is unconditionally 1, the device 200 can generate the expressive vocal level as 1.

Meanwhile, the device 200 may replace the vocal level with a vocal grade and provide it to the user. In this case, the highest vocal level of 8 may be replaced with an SS grade, and the second highest vocal level of 7 may be replaced with a SS grade. Grade S for 6, the 3rd highest vocal level, Grade A for 5, the 4th highest vocal level, Grade B for 5th highest vocal level, Grade C for 4, 6th highest vocal level. The highest vocal level of 3 can be replaced with grade D, the 7th highest vocal level of 2 can be replaced with grade E, and the lowest vocal level of 1 can be replaced with grade F.

Through this process, the device 200 may determine that a user with a high overall vocal level has better ability for the item than a user with a low overall vocal level even if the same output result is obtained for the same item. For example, if user A with a comprehensive vocal level of 6 and user B with a comprehensive vocal level of 3 both output O through the pitch evaluation model, the device 200 compares the vocal level of user A's pitch to user A's comprehensive vocal level. By generating 6, the same as the vocal level of 6, and creating the vocal level of user B's pitch as 3, the same as 3, the overall vocal level of user B, both user A and user B have the same output value through the pitch evaluation model. Even if O is obtained, user A, who has a high overall vocal level, can be judged to have better pitch ability than user B, who has a low overall vocal level.

Additionally, the device 200 can calculate vocal levels corresponding to all items by performing the process for all items.

Figure 7 is an example diagram of vocal levels and vocal grades generated based on output values of an evaluation model corresponding to each item including pitch, tempo, breathing, rhythm, high pitch, expressiveness, and completeness according to an embodiment.

Figure 8 is a flow chart to explain the process of generating user's vocalization attributes according to one embodiment.

Referring to FIG. 8, in step S801, the device 200 combines the value output through the tone evaluation model and the value output through the attribute evaluation model to determine HF, LF, HM, LM, HS, LS, HO, LO. One of the following can be created as a user's vocalization attribute.

For example, as a result of applying the vocal data of the first user to the vocal evaluation artificial intelligence model, if the value output through the tone evaluation model is H and the value output through the attribute evaluation model is O, the device 200 HO can be created as a user's vocalization attribute by combining H, the value output through the tone evaluation model, and O, the value output through the attribute evaluation model.

Meanwhile, when the user's vocalization properties are created, the device 200 can check the characteristics matched to the user's vocalization properties through the vocalization feature database, and in the process of providing the user's vocalization properties to the user's terminal 100, the user The vocalization properties of and the features matched to the corresponding vocalization properties can be provided together. At this time, a vocalization feature database may be provided in the device 200, and each vocalization attribute and a feature corresponding to the vocalization attribute may be matched and stored in the vocalization feature database.

For example, in the vocalization feature database, vocalization attributes and features may be matched as in the example below.

HF - This is the type that breathes with the strongest force and has a high-pitched, husky voice mixed with loud breathing.

LF - Uses the largest amount of breathing and can express husky and strong emotions with a heavy low tone at high volume.

HM - He has the highest level of vocalization among idols and is the type that can handle the highest vocal range, and has a high-toned, clear and solid voice.

LM - This is a type of low-toned, high-volume, clear voice that shows strength in low notes and can digest a wide range of sounds from low to high notes.

HS - This is the type that uses the weakest breathing power, and the high-pitched, clear voice with breathing in a small volume is attractive.

LS - Although the voice is produced with light force, it has a low tone voice with the most breathing and a pure and faint emotional tone.

HO - A clear, high-pitched sound can be produced with the power of small breathing. They often have a clear and refreshing voice.

LO - This is a type of voice that produces a low, low tone. It uses light breathing and uses the vocal cords efficiently to produce a clear sound.

Figure 9 is a diagram related to vocalization properties according to one embodiment.

FIG. 10 is a diagram illustrating an output interface of a user's terminal for providing the user's vocal level and user's vocalization properties to the user's terminal, according to an embodiment.

Specifically, the device 200 may provide a level measurement result including the user's vocal level and corresponding vocal training information to the user's terminal 100, and the user's terminal 100 may provide the user's terminal 100 with the level measurement result including the user's vocal level and the corresponding vocal training information. Level measurement results and corresponding vocal training information can be output through the display.

The item level standard table according to an embodiment of the present invention can be arranged so that the leftmost items have a higher basic vocal ability weight, so guides are provided preferentially starting from the left items necessary for acquiring basic vocal skills. This has the advantage of providing more effective guides and tips.

In addition, the device 200 stores vocalization attributes matched to each of a plurality of users in a database provided in the device 200, so that the proportion of total users according to the user's vocalization attributes and the same gender as the user according to the user's vocalization attributes Vocal attribute information showing the ratio of and the idol's ratio according to the user's vocal attribute can be output on the display of the user's terminal 100. Here, the device 200 may further obtain the user's gender from the user's terminal 100 in order to calculate the ratio of the same gender as the user according to the user's vocalization attributes, and the device 200 may further obtain the user's gender from the user's vocalization attributes. To calculate the ratio of idols according to their attributes, you can communicate with a separate idol database via wired or wireless communication. At this time, the idol database may store information about the idol, including the idol's name, the idol's gender, the idol's position, the idol's vocalization properties, etc. The idol can be a singer of any music genre, and can also be a solo singer or group singer. It can include singers, etc.

Figure 11 is a flowchart for explaining the process of selecting vocal data according to an embodiment.

Referring to FIG. 11, first, in step S1101, when the user's voice is recognized through the user's terminal 100, the device 200 performs a set period of time from the time the user's voice is recognized until the user's voice is not recognized for a set time. Voice data up to that point can be obtained. Here, the setting time is a preset time and may vary depending on the embodiment.

Specifically, the device 200 can recognize the user's voice through the user's terminal 100, and when the user's voice is recognized, the device 200 can store the user's voice data generated based on the recognized user's voice. there is. At this time, the device 200 has a problem of insufficient storage space when storing all the voice data. To solve this problem, the device 200 analyzes the user's voice data generated through the user's terminal 100 and provides the voice data to the user. It is possible to store only voice data analyzed as if the user is singing by distinguishing whether it is singing data or voice data in which the user normally speaks.

To this end, the device 200 can recognize the user's voice through the user's terminal 100, and when the user's voice is recognized through the user's terminal 100, the user's voice is recognized from the time the user's voice is recognized. Voice data can be acquired up to the point when the voice is not recognized for a set period of time. In other words, the set time is a time set to check when the user's voice ends. Here, the file format of the voice data is not limited and may be generated differently depending on the embodiment.

For example, if the setting time is 1 minute, the device 200 recognizes the user's voice through the user's terminal 100, checks the first time point when the user's voice is recognized, and after the first time point If it is confirmed that the voice is not recognized for 1 minute at the second time point, the recognized voice from the first time point to the second time point can be generated as one voice data.

At this time, the device 200 recognizes the user's voice through the user's terminal 100, and when voice data is generated based on the recognized user's voice, the device 200 provides location information of the current location through the user's terminal 100. That is, by obtaining location information where the corresponding voice data was generated, it is possible to match the voice data with the location information where the corresponding voice data was generated. Here, the location information may include location coordinate values, but is not limited thereto.

In step S1102, based on the voice data, the device 200 determines a pitch change in which the pitch difference is greater than the target difference among the pitch changes of the voice included in the voice data, and determines a pitch change in which the pitch difference is greater than the target difference. Confirm, and the number of height changes where the difference in height is greater than the target difference can be generated as the number of confirmations. Here, the target difference is a preset difference value and may vary depending on the embodiment.

Specifically, the device 200 checks the change in frequency of the voice included in the voice data based on the user's voice recognized through the user's terminal 100 and the generated voice data to determine the change in the frequency of the voice included in the voice data. You can check the height change. Additionally, the device 200 may check for pitch changes included in the voice data where the pitch difference is greater than a preset target difference. Additionally, the device 200 may check the number of pitch changes in which the pitch difference is greater than the target difference among the pitch changes included in the voice data and generate the corresponding number as the confirmation number.

That is, the device 200 can generate the number of confirmations by checking the pitch changes included in the voice data that have a pitch difference greater than the target difference. In this case, the pitch change can be confirmed through frequency.

In step S1103, the device 200 may select voice data with a confirmation count greater than the target number as candidate voice data. Here, the target number is a preset number and may vary depending on the embodiment.

Specifically, the device 200 generates voice data based on the user's voice recognized through the user's terminal 100, and determines the target difference among the pitch changes included in the voice data through the frequency of the generated voice data. The number of confirmations can be generated by checking the number of pitch changes with large pitch differences, and by comparing the generated confirmation number with the preset target number, voice data with a confirmation number greater than the preset target number is selected as candidate voice data. can do.

That is, the device 200 may select voice data whose confirmation count is greater than a preset target number as candidate voice data by confirming it as voice data generated during the user's singing process. Additionally, the device 200 may store voice data selected as candidate voice data in a database provided in the device 200 and a memory included in the user's terminal 100. At this time, in the process of storing voice data selected as candidate voice data, the device 200 may also store location information matched to the voice data.

In step S1104, the device 200 may extract one of the candidate voice data and select it as the user's vocal data.

Specifically, the device 200 may select voice data whose confirmation count is greater than a preset target number as candidate voice data, and the device 200 may extract one of the candidate voice data and select it as the user's vocal data. there is. At this time, the process of selecting the user's vocal data from the candidate voice data will be described with reference to FIG. 12.

Through this, the device 200 recognizes the user's voice through the user's terminal 100, generates voice data through the recognized user's voice, and analyzes the voice data to determine whether the voice data is used when the user sings. By distinguishing between data or voice data that the user normally speaks, only data in which the user sings, that is, candidate voice data, is stored and the candidate voice data is used as vocal data, so that the user can sing separately in order to receive a vocal evaluation. Vocal evaluation can be performed using existing stored data without generating data.

Figure 12 is a flowchart for explaining the process of selecting vocal data from candidate voice data according to an embodiment.

Referring to FIG. 12, first, in step S1201, the device 200 can check location information corresponding to candidate voice data.

Specifically, the device 200 may check location information where the candidate voice data that matches the candidate voice data was generated.

In step S1202, the device 200 may check whether there is location information matching the valid location based on the location information. Here, the effective location may be a preset location, and the effective location may include the location of a singing space, for example, the location of a karaoke room, the location of an audition hall, or the location of an agency.

Specifically, the device 200 may check whether there is location information matching a preset valid location based on location information where the candidate voice data that matches the candidate voice data was generated.

If it is confirmed in step S1202 that there is location information matching the valid location, the device 200 may select candidate voice data having location information matching the valid location as the user's vocal data in step S1203.

Specifically, as a result of checking the location information where the candidate voice data matching the candidate voice data was generated, if it is determined that there is location information matching a preset effective location, the device 200 has the location information matching the effective location. Candidate voice data can be selected as the user's vocal data.

For example, there is first candidate voice data, second candidate voice data, third candidate voice data, and fourth candidate voice data, the first location information is matched to the first candidate voice data, and the second candidate voice data The second position information is matched to the third candidate voice data, the third position information is matched to the fourth candidate voice data, and the first effective position is a preset effective position, When there is a second effective location, a third effective location, and a fourth effective location, the device 200 provides first location information matched to the first candidate voice data, and second location information matched to the second candidate voice data. , As a result of checking the third location information matched to the third candidate voice data and the fourth location information matched to the fourth candidate voice data, if it is confirmed that the first location information and the first valid location match, the device 200 ) may select the first candidate voice data having first location information, which is location information matching the first effective location, as the user's vocal data.

If it is determined in step S1202 that there is no location information matching the valid location, the device 200 may randomly extract one of the candidate voice data and select the extracted candidate voice data as the user's vocal data in step S1204.

Specifically, as a result of checking the location information where the candidate voice data matching the candidate voice data was generated, the device 200 determines that there is no position information that matches the preset effective location, randomly extracts one of the candidate voice data and The extracted candidate voice data can be selected as the user's vocal data.

For example, there is first candidate voice data, second candidate voice data, third candidate voice data, and fourth candidate voice data, the first location information is matched to the first candidate voice data, and the second candidate voice data The second position information is matched to the third candidate voice data, the third position information is matched to the fourth candidate voice data, the fourth position information is matched to the fourth candidate voice data, and the second effective position is a preset effective position, If there is a third effective location and a fourth effective location, the device 200 may generate the first location information matched to the first candidate voice data, the second location information matched to the second candidate voice data, and the third candidate voice. As a result of checking the third location information matched to the data and the fourth location information matched to the fourth candidate voice data, the second valid one among the first location information, the second location information, the third location information, and the fourth location information If it is determined that there is no location information matching the location or the third valid location or the fourth valid location, the device 200 selects one of the first candidate voice data, the second candidate voice data, the third candidate voice data, and the fourth candidate voice data. One can be randomly extracted and the extracted candidate voice data can be selected as the user's vocal data.

Through this, the device 200 checks whether there is location information that matches the location of a singing space such as a karaoke room, audition room, entertainment company, etc. through the location information where the candidate voice data was generated, and according to the confirmation result, among the candidate voice data By selecting the user's vocal data, data more suitable for vocal evaluation can be selected.

Figure 13 is a flowchart for explaining a process for correcting a user's vocal level according to an embodiment.

Referring to FIG. 13, first, in step S1301, the device 200 may upload the user's vocal data to the vocal evaluation platform for a target period. Here, the target period is a preset period and may vary depending on the embodiment. Additionally, the vocal evaluation platform is a platform operated for vocal evaluation and may be a platform operated through the device 200.

Specifically, when the user's vocal level is determined through the vocal evaluation artificial intelligence model, the device 200 may upload the user's vocal data to the vocal evaluation platform for a preset target period.

In step S1302, the device 200 may obtain the evaluator's information and the evaluator's evaluation score from the evaluator's terminal who wishes to evaluate the user's vocal data through the vocal evaluation platform. Here, the device 200 can communicate with the evaluator's terminal wired or wirelessly.

Specifically, the evaluator's terminal can access the vocal evaluation platform, the device 200 can provide the user's vocal data to the evaluator's terminal connected to the vocal evaluation platform, and the device 200 can receive vocal data from the evaluator's terminal. The evaluator's evaluation score can be obtained for the evaluator's information and the user's vocal data. Here, the evaluator's information may be information about the evaluator including the evaluator's name, evaluator's occupation, evaluator's gender, and evaluator's age, and the evaluator's evaluation score may be the score by which the evaluator evaluates the user's vocal data. . At this time, the evaluator's evaluation score can be given as a number from 1 to 8, with 8 being given when the song satisfies all the elements and gives a flawless and solid feel, and 7 means that the more you sing, the more you listen to the song. , it can be given when you instantly feel that you are good at it just by listening to it, 6 is a level where you can feel a lot of practice, and can be given when you show overall stable singing, and 5 is a song or K-pop feeling that is good but somewhat unstable. 4 can be given when a feeling like K-pop or K-pop begins to appear, 3 can be given when a person has the minimum basic skills but has difficulty feeling like K-pop or K-pop, 2 can be given when can be given when the song functions as a song but the basic skills are still lacking, and 1 can be given when the song is closer to speaking than singing. However, it is not limited to this.

In step S1303, the device 200 can determine whether the evaluator is the original composer through the evaluator's information.

Specifically, when the device 200 obtains the evaluator's evaluation score for the evaluator's information and the user's vocal data from the evaluator's terminal, the device 200 compares the song information obtained based on the user's vocal data and the evaluator's information to determine the evaluator's evaluation score. You can check whether it is the original composer or not.

If the evaluator is confirmed to be the original singer in step S1303, the device 200 may apply a first weight to the evaluator's evaluation score in step S1304.

Specifically, if the evaluator who evaluated the user's vocal data is confirmed to be the original composer of the song obtained based on the user's vocal data, the device 200 determines that the evaluator is familiar with the song and adds the evaluator's evaluation score to the The first weight, which is a high weight, can be applied.

If it is confirmed in step S1303 that the evaluator is not the original songwriter, in step S1305, the device 200 can determine whether the evaluator has a job related to music through the evaluator's information. Here, occupations related to music can be set in advance and can be singers, lyricists, composers, arrangers, record planners, music directors, music teachers, vocal trainers, etc.

Specifically, if it is confirmed that the evaluator who evaluated the user's vocal data is not the original composer of the song obtained based on the user's vocal data, the device 200 verifies the occupation of the evaluator through the evaluator's information, and determines the evaluator's occupation. You can determine whether or not your job is related to music.

If it is confirmed that the evaluator has an occupation related to music in step S1305, the device 200 may apply a second weight smaller than the first weight to the evaluator's evaluation score in step S1306.

Specifically, the device 200 determines that if the evaluator who evaluated the user's vocal data is not the original composer of the song obtained based on the user's vocal data, but is confirmed to have an occupation related to music, the evaluator will select the song rather than the original composer. Although they know less, it is determined that they still know a lot about the song, so a second weight smaller than the first weight can be applied to the evaluator's evaluation score.

If it is confirmed in step S1305 that the evaluator does not have a job related to music, in step S1307, the device 200 may apply a third weight smaller than the second weight to the evaluator's evaluation score.

Specifically, if it is confirmed that the evaluator who evaluated the user's vocal data is not the original composer of the song obtained based on the user's vocal data and does not have a job related to music, the device 200 determines that the evaluator is not the original composer and music liaison. A third weight smaller than the second weight can be applied to the evaluator's evaluation score by determining that the person knows less about the song than the related job.

In step S1308, the device 200 may correct the user's vocal level based on the weighted evaluator's evaluation score.

Specifically, the device 200 may acquire the evaluator's information and the evaluator's evaluation score from the evaluator's terminal who wishes to evaluate the user's vocal data through the vocal evaluation platform during the target period, and the device 200 may obtain the evaluator's information. A weight can be applied to the evaluator's evaluation score, and the device 200 can correct the user's vocal level based on the weighted evaluator's evaluation score obtained during the target period.

Through this, the device 200 has the effect of correcting the user's vocal level by considering the evaluator's evaluation score and the user's vocal level obtained through the vocal evaluation artificial intelligence model.

FIG. 14 is a flowchart illustrating a process for providing audition content in progress in an audition available area to a user's terminal according to an embodiment.

Referring to FIG. 14, first, in step S1401, the device 200 may check location information corresponding to the user's vocal data and select an audition area based on the location information.

Specifically, the device 200 may check location information matched to the user's vocal data and, based on the location information, select an area containing the location information as an audition available area. Here, the auditionable area may be a dong unit, a district unit, a city unit, a province unit, or other units.

Additionally, the device 200 can communicate wired or wirelessly with an audition recruitment site, and the audition recruitment site can communicate wired or wirelessly with terminals of a plurality of agencies. In other words, the agency can produce first content, which is content that introduces the area where the audition will be held and the first audition to be held in that area, and upload it to the audition recruitment site, and the device 200 can generate the first content that is uploaded to the audition recruitment site. By checking the content, you can see which auditions are being held in which region. To this end, the device 200 may be connected to a server that operates an audition recruitment site.

In step S1402, the device 200 may control the first content to be played and displayed on the user's terminal 100.

Specifically, the device 200 confirms that the first content is uploaded to the audition recruitment site, and when it confirms through the first content that the corresponding audition is held in an audition available area, the device 200 connects the server to the server that operates the audition recruitment site. Information on the first content can be obtained from the user, and the information on the first content can be transmitted to the user's terminal 100. Through this, the first content is played on the user's terminal 100, and the user's terminal 100 The first content can be controlled to be displayed on the screen.

In step S1403, when the second content, which is content uploaded by a first agency among agencies located within an audition available area, is uploaded to the audition recruitment site, the device 200 continues the first content on the user's terminal 100. 2 You can control how content is played and displayed.

Specifically, when the device 200 confirms that the second content uploaded by the first agency located within the audition available area is uploaded to the audition recruitment site, the device 200 obtains information about the second content from the server operating the audition recruitment site, and 2 Content information can be transmitted to the user's terminal 100, and through this, when playback of the first content is completed on the user's terminal 100, the second content is played subsequently, and the screen of the user's terminal 100 is displayed. It is possible to control the second content to be displayed. For this purpose, the audition recruitment site may further store information about the agency that matched the content uploaded to the audition recruitment site and uploaded the content.

The device 200 plays the first content on the user's terminal 100, and if the second content has been uploaded, controls the second content to be played after the first content, so that the user is introduced to various auditions held in audition available areas. You can make it accessible.

In step S1404, if the third content, which is content uploaded by a second agency among agencies located within the audition available area, is uploaded to the audition recruitment site, the device 200 continues the second content in the user's terminal 100. 3 You can control how content is played and displayed.

Specifically, when the device 200 confirms that the third content uploaded by the second agency located within the audition available area is uploaded to the audition recruitment site, the device 200 obtains information about the third content from the server operating the audition recruitment site, and 3 The information on the content can be transmitted to the user's terminal 100, and through this, when the playback of the second content is completed on the user's terminal 100, the third content is played subsequently, and the screen of the user's terminal 100 is displayed. You can control the display of third content.

In other words, the device 200 can provide the user with information on various agencies located within the audition available area by providing content uploaded by various agencies located within the audition available area, and plays the second and third contents of the agencies. It can show diversity and various opportunities.

In step S1405, the device 200 may control the user's terminal 100 to play and display the first content following the third content.

Specifically, the device 200 can control the first content to be played again when the third content is completed on the user's terminal 100 so that the first content is displayed again on the screen of the user's terminal 100.

The device 200 plays the first, second, and third contents in order on the user's terminal 100, and sequentially shows the user information about auditions to be held in audition available areas and introduction of agencies located in audition available areas. Therefore, by sequentially playing the first, second, and third contents, audition information and the introduction of the agency can be gradually delivered to the user. This approach maintains the user's interest while delivering information sequentially and You can become familiar with it.

In step S1406, the device 200 may control the user's terminal 100 to play and display the second content and the third content together following the first content.

Specifically, when the playback of the first content is completed on the user's terminal 100, the device 200 then plays the second content and the third content together, and displays the second content and the third content on the screen of the user's terminal 100. You can control the content to be split and displayed.

The device 200 allows the user to access various information without missing it because the content provided through the user's terminal 100 is visually and effectively delivered, and the user receives information about auditions to be held in the audition available area and locations located in the audition available area. You can increase interest in the agency's introduction and get more opportunities to participate in auditions. Additionally, you can easily access information about entertainment companies, which can help you make choices and decisions about musical activities.

Figure 15 is an exemplary diagram of the configuration of a device according to an embodiment.

Device 200 according to one embodiment includes a processor 210 and memory 220. The device 200 according to one embodiment may be the server or terminal described above. The processor 210 may include at least one device described above with reference to FIGS. 1 to 14 or may perform at least one method described with reference to FIGS. 1 to 14 . The memory 220 may store information related to the above-described method or store a program implementing the above-described method. Memory 220 may be volatile memory or non-volatile memory.

The processor 210 can execute programs and control the device 200. The code of the program executed by the processor 210 may be stored in the memory 220. The device 200 is connected to an external device (eg, a personal computer or a network) through an input/output device (not shown) and can exchange data.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

Although the embodiments have been described with limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the following claims.

Claims (3)

In the vocal data collection using the user's terminal and the vocal evaluation method using the same, which is performed by the device,
Obtaining the user's vocal data from the user's terminal;
Obtaining song information based on the user's vocal data;
Applying the user's vocal data and the song information to a vocal evaluation artificial intelligence model to output the user's evaluation result data;
calculating the user's vocal level based on the user's evaluation result data;
Based on the user's evaluation result data, generating a user's vocalization attribute as one of HF, LF, HM, LM, HS, LS, HO, and LO; and
Providing the user's vocal level and the user's vocalization attributes to the user's terminal,
The vocal evaluation artificial intelligence model is
Includes an item evaluation model and a speech attribute evaluation model;
The item evaluation model above is
The first distance, which is the distance between sounds in the first setting section, and the first height, which is the height of the sound, are determined through the song information, and a second setting corresponding to the first setting section is determined through the user's vocal data. Determine the second distance, which is the distance between sounds in the section, and the second height, which is the height of the sound, compare the first distance and the second distance, and compare the first height and the second height. If it is confirmed that the difference between the first distance and the second distance is within a preset first reference difference, and that the difference between the first height and the second height is within a preset second reference difference, output O, Output A pitch evaluation model that
The standard BPM is confirmed through the song information, a section outside the standard BPM is created through the user's vocal data, and if it is confirmed that the section outside the standard BPM is shorter than the preset standard time, O is output, and the standard BPM is generated. A tempo evaluation model that outputs
The number of times the vibrato is included is determined through the user's vocal data, and if it is confirmed that the vibrato satisfies the first standard number of times set in advance, O is output, and it is confirmed that the vibrato does not satisfy the first standard number of times. A respiratory control evaluation model that outputs
The reference rhythm is confirmed through the song information, the number of times the bending and vibration corresponding to the reference rhythm is included is determined through the user's vocal data, and the number of times the bending and vibration corresponding to the reference rhythm is included is calculated in advance. Rhythm evaluation that outputs O when it is confirmed that the set second standard number of times was satisfied, and outputs X when it is confirmed that the number of bending and vibrations corresponding to the standard rhythm was included did not satisfy the second standard number. Model,
Confirm the user's vocal range through the user's vocal data, generate a reference treble height based on the user's vocal range, and output O when it is confirmed that the change in sound in the reference treble height is smaller than the preset reference change. , a treble stability evaluation model that outputs
Generating the number of changes in diction's accent through the user's vocal data, generating the number of changes in sound texture through the user's vocal data, and generating the number of changes in volume size through the user's vocal data, and generating the diction If it is confirmed that the number of expressions of the user, which is the number of times the stress change, the number of changes in the texture of the sound, and the number of changes in the volume, satisfy the preset third standard number, O is output, and the user's expression An expression evaluation model that outputs X when it is confirmed that the number of times does not meet the third standard number, and
When O is output in all of the pitch evaluation model, the tempo evaluation model, the breathing control evaluation model, the rhythm evaluation model, the treble stability evaluation model, and the expressiveness evaluation model, an impactful portion through the user's vocal data It includes a completeness evaluation model that determines whether there is a part with the impact, outputs O if it is confirmed that there is a part with the impact, and outputs X if it is confirmed that there is no part with the impact,
The vocalization attribute evaluation model is
The user's average voice frequency is checked through the user's vocal data, and if it is confirmed that the user's average voice frequency is higher than the preset reference frequency, H is output, and the user's average voice frequency is lower than the reference frequency. a tone evaluation model that outputs L if confirmed to be identical, and
The user's average voice decibel is confirmed through the user's vocal data, and whether the user's vocal cords are grounded through the user's vocal data, and the user's average voice decibel is greater than a preset reference decibel and the user's If it is confirmed that the vocal cords are grounded, M is output, and if the user's average voice decibel is greater than the reference decibel and it is confirmed that the user's vocal cords are not grounded, F is output, and the user's average voice decibel is less than the reference decibel. If it is confirmed that the user's vocal cords are grounded, O is output, and if the user's average voice decibel is less than or equal to the reference decibel and it is confirmed that the user's vocal cords are not grounded, an attribute evaluation model outputs S. Contains,
The step of calculating the user's vocal level based on the user's evaluation result data,
calculating the user's overall vocal level, and
Including the step of calculating the vocal level for each item,
The step of calculating the user's comprehensive vocal level is
Checking the total number of O output through the item evaluation model, and
Generating the user's overall vocal level by applying +1 to the total number of O's,
The steps for calculating the vocal level for each item are:
When the user's comprehensive vocal level is generated, checking whether the output value of the item evaluation model corresponding to the confirmation item for which the vocal level is to be calculated is O or X;
If the output value of the item evaluation model corresponding to the confirmation item is confirmed to be O, assigning the vocal level of the confirmation item as a vocal level with the same level value as the user's overall vocal level,
When it is confirmed that the output value of the item evaluation model corresponding to the confirmation item is X, determining whether the confirmation item is complete;
If the confirmation item is confirmed to be complete, assigning the vocal level of the confirmation item as a vocal level of a level value generated by applying -3 to the user's overall vocal level, and
When it is confirmed that the confirmation item is not complete, assigning the vocal level of the confirmation item as a vocal level of a level value generated by applying -2 to the user's overall vocal level,
The step of generating the user's vocalization attributes based on the user's evaluation result data,
A step of combining the value output through the tone evaluation model and the value output through the attribute evaluation model to generate one of HF, LF, HM, LM, HS, LS, HO, and LO as the user's vocalization attribute. ,
Vocal data collection using the user's terminal and vocal evaluation method using the same. delete According to paragraph 1,
Obtaining the user's vocal data from the user's terminal;
When the user's voice is recognized through the user's terminal, acquiring voice data from the time the user's voice is recognized to the time when the user's voice is not recognized for a preset time,
Based on the voice data, confirming a pitch change whose pitch difference is greater than a preset target difference among the pitch changes included in the voice data, and generating the number of pitch changes greater than the target difference as the number of confirmations,
Selecting voice data whose number of confirmations is greater than a preset target number as candidate voice data;
Extracting one of the candidate voice data and selecting it as the user's vocal data,
The step of extracting one of the candidate voice data and selecting it as the user's vocal data is,
Confirming location information corresponding to the candidate voice data,
Based on the location information, checking whether there is location information matching a preset valid location,
When it is confirmed that there is location information matching the effective location, selecting candidate voice data having location information matching the effective location as the user's vocal data, and
When it is determined that there is no location information matching the effective location, randomly extracting one of the candidate voice data and selecting the extracted candidate voice data as the user's vocal data,
Vocal data collection using the user's terminal and vocal evaluation method using the same.