CN114694678A - Sound quality detection model training method, sound quality detection method, electronic device, and medium - Google Patents

Abstract

The present application discloses a sound quality detection model training method, a sound quality detection method, equipment and a computer-readable storage medium. The sound quality detection model training method includes: obtaining initial training audio and corresponding average opinion score labels; Interference filtering processing of endpoint detection to obtain training audio; feature extraction processing of training audio to obtain training features; input training features to the initial model to obtain the corresponding training sound quality detection results; use the training sound quality detection results and the average opinion score label to calculate loss value, and use the loss value to adjust the model parameters of the initial model; when it is detected that the training completion conditions are met, the adjusted initial model is determined as the sound quality detection model; the obtained sound quality detection model can be used without pure and clean audio. , to accurately assess the quality of the audio under test.

Description

Sound quality detection model training method, sound quality detection method, electronic equipment and medium

technical field

The present application relates to the technical field of audio processing, and in particular, to a training method for a sound quality detection model, a sound quality detection method, an electronic device, and a computer-readable storage medium.

Background technique

At present, the method of PESQ (that is, perceptual evaluation of speech quality, perceptual evaluation of speech quality) is usually used to detect the audio quality, and a detection result representing the quality of the sound quality is obtained. The PESQ method is usually aimed at the audio of VOIP (Voice over Internet Protocol, voice over IP) network communication, and can evaluate the audio time misalignment and spectrum distortion caused by the influence of frame loss and jitter during the network transmission of the audio signal. . To calculate the PESQ score, it is necessary to prepare pure and clean audio corresponding to the noise-containing frequency. This sound quality evaluation method is called the reference sound quality evaluation. In applications, pure and clean audio is difficult to obtain, making it difficult to detect the sound quality of most audios.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present application is to provide a sound quality detection model training, a sound quality detection method, an electronic device and a computer-readable storage medium, so as to accurately evaluate the quality of the audio to be tested.

In order to solve the above technical problems, in the first aspect, the present application provides a method for training a sound quality detection model, including:

Obtaining the initial training audio and the corresponding average opinion score label; the average opinion score label is used to represent the average sound quality evaluation parameter obtained after a plurality of evaluation objects perform sound quality evaluation on the initial training audio;

Carrying out the non-human voice filtering process based on voice endpoint detection to the initial training audio to obtain the training audio;

Perform feature extraction processing on the training audio to obtain training features;

Input the training feature into the initial model to obtain the corresponding training sound quality detection result;

Calculate a loss value using the training sound quality detection result and the average opinion score label, and use the loss value to adjust the model parameters of the initial model;

When it is detected that the training completion condition is satisfied, the adjusted initial model is determined as a sound quality detection model.

Optionally, the process of obtaining the average opinion score label includes:

playing the initial training audio to each of the evaluation objects;

receiving initial sound quality data obtained after each of the evaluation objects performs sound quality evaluation on the initial training audio;

The mean opinion score label is generated using each of the initial voice quality data.

Optionally, generating the average opinion score label using each of the initial sound quality data includes:

Perform averaging processing on each of the initial sound quality data to obtain a first score label;

Inputting the initial training audio into an audio defect detection model to obtain a defect detection result; wherein the audio defect detection model is used to detect audio defects that can affect auditory experience;

generating a second score label based on the defect detection result;

The average opinion score label is generated using the first score label and the second score label.

Optionally, performing feature extraction processing on the training audio to obtain training features, including:

Based on the maximum sampling rate perceivable by the human ear, the training audio is resampled to obtain intermediate data;

Performing sliding window framing based on a preset window length on the intermediate data to obtain a plurality of audio frames;

Feature extraction processing is performed on each of the audio frames to obtain the training features.

Optionally, performing non-human voice filtering processing based on voice endpoint detection on the initial training audio to obtain training audio, including:

The voice endpoint detection is carried out to the initial training audio, and the voice endpoint time is obtained;

According to the voice endpoint time, the initial training audio is segmented to obtain a plurality of audio segments, and the non-human voice frequency range in the audio segment is removed to obtain the human voice frequency range;

The training audio is obtained by splicing the human voice frequency bands.

Optionally, the initial model includes a convolutional neural network, a long short-term memory network, a fully connected layer and an average pooling layer;

The described training features are input into the initial model to obtain corresponding training sound quality detection results, including:

Inputting the training features into the convolutional neural network to obtain training intermediate features;

Inputting the training intermediate feature into the long short-term memory network to obtain the training initial detection result;

Inputting the training initial detection result into the fully connected layer to obtain the training intermediate detection result;

The training intermediate detection result is input into the average pooling layer to obtain the training sound quality detection result.

Optionally, the obtaining of the initial training audio and the corresponding average opinion score label includes:

According to a preset batch size, obtain a batch of a plurality of the initial training audio and the average opinion score label from the training data set;

Correspondingly, calculating the loss value using the training sound quality detection result and the average opinion score label includes:

When the training sound quality detection results corresponding to all the initial training audios corresponding to a batch are obtained, the loss value is obtained by using the training sound quality detection results, the average opinion score label and the training intermediate detection results in the batch .

Optionally, the loss value is obtained by using the training sound quality detection result, the average opinion score label and the training intermediate detection result in the batch, including:

according to

obtain the loss value;

为所述训练音质检测结果，所述

为所述训练特征中的第t帧在所述训练中间检测结果中对应的值，所述α为预设权重。Wherein, the loss is the loss value, the S is the preset batch size, the T _S is the number of frames corresponding to the training feature, the M _S is the average opinion score label, and the stated

For the training sound quality detection result, the

is the value corresponding to the t-th frame in the training feature in the intermediate detection result of the training, and the α is a preset weight.

Optionally, it is detected that the training completion condition is met, including:

judging whether the loss value is less than a preset threshold;

If so, it is determined that the training completion condition is satisfied.

In a second aspect, the present application also provides a sound quality detection method, including:

Get the initial audio to be tested;

Carrying out non-human voice filtering processing based on voice endpoint detection to the initial audio to be tested to obtain audio to be tested;

Perform feature extraction processing on the audio to be tested to obtain features to be tested;

The feature to be tested is input into a sound quality detection model to obtain a sound quality detection result corresponding to the initial to-be-tested audio.

Optionally, the sound quality detection model includes a convolutional neural network, a long short-term memory network, a fully connected layer and an average pooling layer;

The inputting the feature to be measured into the sound quality detection model to obtain the sound quality detection result corresponding to the initial to-be-measured audio, including:

Inputting the feature to be tested into the convolutional neural network to obtain an intermediate feature to be tested;

Inputting the intermediate feature to be tested into the long short-term memory network to obtain an initial detection result;

Inputting the initial detection result into the fully connected layer to obtain an intermediate detection result;

The intermediate detection result is input into the average pooling layer to obtain the sound quality detection result.

In a third aspect, the present application also provides an electronic device, including a memory and a processor, wherein:

the memory for storing computer programs;

The processor is configured to execute the computer program to implement the above-mentioned method for training a sound quality detection model, and/or the above-mentioned method for sound quality detection.

In a fourth aspect, the present application further provides a computer-readable storage medium for storing a computer program, wherein, when the computer program is executed by a processor, the above-mentioned training method for a sound quality detection model is implemented, and/or the above-mentioned sound quality Detection method.

The training method for a sound quality detection model provided by this application is to obtain initial training audio and corresponding average opinion score labels; perform interference filtering processing based on voice endpoint detection on the initial training audio to obtain training audio; perform feature extraction processing on the training audio to obtain Training features; input the training features into the initial model to obtain the corresponding training sound quality detection results; use the training sound quality detection results and the average opinion score label to calculate the loss value, and use the loss value to adjust the model parameters of the initial model; when it is detected that the training completion conditions are met , the adjusted initial model is determined as the sound quality detection model.

It can be seen that the method utilizes the mean opinion score (MOS) as the initial training audio and the label of the training audio. Mean Opinion Scores were assessed by a large audience for the quality of audio of sentences read aloud by a male or female speaker, transmitted over a communication circuit. Listeners rated each sentence on the following scale: (1) very poor (2) poor (3) fair (4) good (5) very good, MOS is an arithmetic measure of individual scores for all listeners, ranging from 1 (worst) to 5 (best). The resulting MOS labels can accurately characterize the quality of the sentence audio in the initial training audio. Through speech endpoint detection, the part of non-human voice audio can be used as interference and filtered, and only the part of human voice can be retained as training audio. After the training features are obtained through feature extraction, the initial model is used for quality detection, and the loss value is calculated using the obtained training sound quality detection results and the average opinion score label. The obtained training sound quality detection results are as close as possible to the MOS labels. After the training is completed, the adjusted initial model can be determined as a sound quality detection model. The resulting sound quality detection model can accurately evaluate the quality of the audio under test without having pure clean audio.

In addition, the present application also provides a sound quality detection method, an electronic device and a computer-readable storage medium, which also have the above beneficial effects.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that are used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only It is an embodiment of the present application. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without any creative effort.

1 is a schematic diagram of a hardware composition framework to which a sound quality detection model training method provided by an embodiment of the present application is applicable;

2 is a schematic diagram of a hardware composition framework to which another sound quality detection model training method provided by an embodiment of the present application is applicable;

3 is a flowchart of a method for training a sound quality detection model provided by an embodiment of the present application;

4 is a flowchart of a sound quality detection method provided by an embodiment of the present application;

FIG. 5 is a flowchart of another sound quality detection method provided by an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

For ease of understanding, the method for training a sound quality detection model provided by the embodiments of the present application and/or the hardware composition framework used in the solution corresponding to the sound quality detection method are introduced. Please refer to FIG. 1 , which is a schematic diagram of a hardware composition framework to which a method for training a sound quality detection model provided by an embodiment of the present application is applicable. The electronic device 100 may include a processor 101 and a memory 102 , and may further include one or more of a multimedia component 103 , an information input/information output (I/O) interface 104 and a communication component 105 .

Wherein, the processor 101 is used to control the overall operation of the electronic device 100 to complete the training method of the sound quality detection model, and/or all or part of the steps in the sound quality detection method; the memory 102 is used to store various types of data to support Operation of the electronic device 100, such data may include, for example, instructions for any application or method operating on the electronic device 100, as well as application-related data. The memory 102 may be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (Erasable) Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (Read-Only Memory) , ROM), magnetic memory, flash memory, magnetic disk or optical disk one or more. In this embodiment, the memory 102 stores at least programs and/or data for implementing the following functions:

Get the initial training audio and the corresponding mean opinion score label;

Perform non-human voice filtering processing based on voice endpoint detection on the initial training audio to obtain training audio;

Perform feature extraction processing on the training audio to obtain training features;

Input the training features into the initial model to obtain the corresponding training sound quality detection results;

Use the training sound quality detection results and the average opinion score label to calculate the loss value, and use the loss value to adjust the model parameters of the initial model;

When it is detected that the training completion condition is satisfied, the adjusted initial model is determined as the sound quality detection model.

and / or,

Get the initial audio to be tested;

Perform non-human voice filtering processing based on voice endpoint detection on the initial audio to be tested to obtain audio to be tested;

Perform feature extraction processing on the audio to be tested to obtain features to be tested;

Input the feature to be tested into the sound quality detection model, and obtain the sound quality detection result corresponding to the audio to be initially tested.

Multimedia components 103 may include screen and audio components. Wherein the screen can be, for example, a touch screen, and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 102 or transmitted through the communication component 105 . The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 104 provides an interface between the processor 101 and other interface modules, and the above-mentioned other interface modules may be a keyboard, a mouse, a button, and the like. These buttons can be virtual buttons or physical buttons. The communication component 105 is used for wired or wireless communication between the electronic device 100 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC for short), 2G, 3G or 4G, or one or a combination of them, so the corresponding communication component 105 may include: Wi-Fi parts, Bluetooth parts, NFC parts.

The electronic device 100 may be implemented by one or more Application Specific Integrated Circuit (ASIC for short), Digital Signal Processor (DSP for short), Digital Signal Processing Device (DSPD for short), or Programmable Logic Device (PLD for short), Field Programmable Gate Array (FPGA for short), controller, microcontroller, microprocessor or other electronic components to implement, used to perform sound quality detection model training method .

Of course, the structure of the electronic device 100 shown in FIG. 1 does not constitute a limitation on the electronic device in the embodiments of the present application. In practical applications, the electronic device 100 may include more or less components than those shown in FIG. 1 , or a combination of certain parts.

It can be understood that the embodiment of the present application does not limit the number of electronic devices, which may be a method for training a sound quality detection model and/or a method for sound quality detection jointly completed by multiple electronic devices. In a possible implementation, please refer to FIG. 2 , which is a schematic diagram of a hardware composition framework to which another sound quality detection model training method provided by an embodiment of the present application is applicable. As can be seen from FIG. 2 , the hardware composition framework may include: a first electronic device 11 and a second electronic device 12 , which are connected through a network 13 .

In this embodiment of the present application, the hardware structures of the first electronic device 11 and the second electronic device 12 may refer to the electronic device 100 in FIG. 1 . That is, it can be understood that there are two electronic devices 100 in this embodiment, and the two perform data interaction. Further, the embodiment of the present application does not limit the form of the network 13, that is, the network 13 may be a wireless network (such as WIFI, Bluetooth, etc.) or a wired network.

The first electronic device 11 and the second electronic device 12 can be the same electronic device, for example, the first electronic device 11 and the second electronic device 12 are both servers; they can also be different types of electronic devices, for example, the first electronic device 11 and the second electronic device 12 are servers. The device 11 may be a smart phone or other smart terminals, and the second electronic device 12 may be a server. In a possible implementation, a server with strong computing power may be used as the second electronic device 12 to improve data processing efficiency and reliability, thereby improving the processing efficiency of the training of the sound quality detection model. At the same time, a smartphone with low cost and wide application range is used as the first electronic device 11 to realize the interaction between the second electronic device 12 and the user. It can be understood that the interaction process can be as follows: the user obtains the initial training audio on the smartphone and gives the corresponding average opinion score label, the smartphone sends the initial training audio and the average opinion score label to the server, and the server uses the initial training audio and the average opinion score label. Training audio and MOS label training to obtain a sound quality detection model. The server sends the sound quality detection model to the smartphone for sound quality detection on the smartphone.

Alternatively, the sound quality detection model is deployed on the server, and the smartphone can interact with the user to obtain the initial audio under test and send it to the server. The server uses the sound quality detection model to detect the initial audio to be tested, obtains the corresponding sound quality detection result, and sends the sound quality detection result to the smartphone for output to the user.

Please refer to FIG. 3 , which is a schematic flowchart of a method for training a sound quality detection model according to an embodiment of the present application. The method in this embodiment includes:

S101: Obtain the initial training audio and the corresponding average opinion score label.

The Mean Opinion Score, or Mean Opinion Score, MOS, is used by a large audience to assess the quality of audio of sentences read aloud by male or female speakers, transmitted over a communication circuit. Listeners rated each sentence on the following scale: (1) very poor (2) poor (3) fair (4) good (5) very good, MOS is an arithmetic measure of individual scores for all listeners, ranging from 1 (worst) to 5 (best). This evaluation method is widely used in the subjective evaluation of audio quality, however, the process of subjective evaluation is time-consuming and labor-intensive. Therefore, the PESQ (perceptual evaluation of speech quality, perceptual evaluation of speech quality) method is widely used in the automatic detection of audio quality (or called objective evaluation) to improve the efficiency of audio quality detection. However, PESQ is a reference sound quality evaluation, and the audio to be tested can only be detected when there is pure and clean audio of the audio to be tested (which can be regarded as lossless audio), which makes the detection range limited.

In this application, the existing initial training audio and corresponding MOS tags are used to form training data, and a sound quality detection model is obtained by training, so that automatic sound quality detection can be performed without pure and clean audio, so as to improve the detection efficiency and expand the detection range. Specifically, the initial training audio refers to directly obtained training data, which may have evidence or multiple speech sentences. The average opinion score label refers to the MOS score label obtained by manually MOS scoring the speech part in the initial training audio, and is used to represent the average sound quality evaluation parameter obtained by multiple evaluation objects after the sound quality evaluation of the initial training audio. The initial training audio and the corresponding MOS label can be prepared in advance, or when the sound quality detection model needs to be trained, the initial training audio can be temporarily selected and the user can score it in MOS to obtain the corresponding MOS label.

In one embodiment, the mean opinion score label may be generated at acquisition time. Specifically, the initial training audio can be played to each evaluation object, for example, the initial training audio and a playback control instruction can be sent to the electronic device used by each evaluator. Receive initial sound quality data obtained after each evaluation object performs sound quality evaluation on the initial training audio, and the initial sound quality data may be generated and sent by electronic devices used by each evaluator. A mean opinion score label is generated using each of the initial sound quality data.

This embodiment does not limit the specific manner of generating the average opinion score. For example, each initial sound quality data may be averaged. In another embodiment, in order to improve the reliability of the average opinion score label, each initial sound quality data may be averaged to obtain the first score label. In addition, the initial training audio can also be input into the audio defect detection model to obtain the defect detection result; the audio defect detection model is used to detect audio defects that can affect the auditory experience. (Generated because the mouth is too close to the microphone, manifested as po sound, hu sound, grunt, etc.), hissing/tooth sound (presented as hissing sound, eating chi sound), current sound (caused by abnormal hardware circuit) The noise caused by it is expressed as sizzling/humming sound), click sound (expressed as thorn, click, and stuck sound), popping sound (called clip, due to sound popping, clipping, etc.) It is also easy to produce after the voice and accompaniment are mixed), ambient noise, and stuttering (expressed as short-term interruption of vocals, poor connection before and after, or obvious network transmission swallow). The audio defect detection model can identify audio defects in the initial training audio, which may be the number of types of audio defects, the number of times each type of audio defects occurs, and the like. Based on the defect detection result, a second score label may be generated. In one embodiment, a full score of 5 points may be set, and points may be deducted as appropriate according to the defect detection result to obtain a second score label, and the first score label and the second score label may be used. Score labels generate mean opinion score labels.

S102: Perform non-human voice filtering processing based on voice endpoint detection on the initial training audio to obtain training audio.

It is understandable that the reader usually speaks multiple sentences in the recording of the initial training audio, often with speech-free time intervals or other non-vocal audio between sentences. When MOS scoring, users only evaluate the voice part, ignoring non-human voice audio. Therefore, during the training of the sound quality detection model, the non-human voice audio in the initial training audio should be removed to avoid interference with the training of the sound quality detection model. In this embodiment, the voice endpoint detection method can be used to identify the start time position and the end time position of the human voice, and then filter the non-human voice part in the initial training audio to obtain the training audio. Voice endpoint detection is Voice Activity Detection, VAD, which is able to identify periods of silence in the audio signal.

Specifically, in one embodiment, the generation process of the training audio includes:

Step 11: Perform voice endpoint detection on the initial training audio to obtain the voice endpoint time.

Step 12: Segment the initial training audio according to the voice endpoint time to obtain a plurality of audio segments, and remove the non-human voice frequency range in the audio segment to obtain the human voice frequency range.

Step 13: Splicing vocal frequency bands to obtain training audio.

In this embodiment, the voice endpoint detection can identify the start time and end time of the voice audio (ie, human voice audio), which are the voice endpoint time. The initial training audio is segmented according to the voice endpoint time to obtain audio segments, which include a human voice band and a non-human voice band, and the two audio segments appear alternately in turn. The non-human voice frequency bands are removed, and the human voice frequency bands are retained. Exemplarily, the human voice frequency range and the non-human voice frequency range appear alternately, so the start time and the end time in the voice endpoint time also appear alternately. Along the chronological sequence, the audio segment between the adjacent start time and the end time is determined as the human voice frequency range, and the audio segment between the adjacent end time and the start time is determined as the non-human voice frequency range. After the audio segment classification is completed, the non-human voice band is removed, the human voice band is retained, and then spliced to obtain the final training audio. The non-human voice segment may be a blank audio segment, or may be an audio segment recorded with non-human voices such as background sounds.

S103: Perform feature extraction processing on the training audio to obtain training features.

In order to enable the initial model to learn how to detect the sound quality of the audio more efficiently, feature extraction is performed on the training audio to obtain corresponding training features, so as to better characterize the sound quality characteristics of the training audio. The specific manner of feature extraction is not limited. In one embodiment, the training feature may be in the form of an image, for example, a spectrogram (Spectrogram), a Mel-Frequency Spectrum (Mel-Frequency Spectrum) or other spectrogram forms. In another embodiment, in order to evaluate the signal over a wider frequency range, the training audio can also be resampled to make its signal frequency better.

Specifically, the generation process of training features may include:

Step 21: Based on the maximum sampling rate perceivable by the human ear, resample the training audio to obtain intermediate data.

Step 22: Perform sliding window segmentation based on a preset window length on the intermediate data to obtain multiple audio frames.

Step 23: Perform feature extraction processing on each audio frame to obtain training features.

The frequency range of the signal that the human ear can hear is relatively fixed, usually in the range of 20Hz-20000Hz. The sampling frequency is usually 2 times the signal frequency, so the maximum sampling rate perceivable by the human ear can be determined. It is understandable that due to the different hearing ranges of different people, the upper limit of the human ear frequency range of some people can reach 22000Hz, so the maximum sampling rate perceivable by the human ear can be higher than the upper limit sampling rate of the ordinary human ear frequency range. Size is not limited. Exemplarily, 48 kHz may be selected as the maximum sampling rate perceivable by the human ear. Through resampling, the audio can be trained to broaden the frequency domain range, and the corresponding intermediate data can be obtained.

Sliding window framing refers to the process of using a preset analysis window to sample audio frames in time sequence. The analysis window can be Hann, Hamming, Blackman-Harris window. -harris), etc., the preset window length is the width of the analysis window, which is not specifically limited. For example, when the sampling frequency is 48kHz, the preset window length can be 21.3ms. After each sampling of the analysis window, it slides back a certain distance for re-sampling, and the sliding distance is the frame shift. The specific size of the frame shift is not limited, for example, it can be half of the window length.

After the audio frame is obtained, feature extraction processing such as mel spectrum extraction and spectrogram extraction is performed on the audio signal to obtain the audio frame feature corresponding to a single audio frame, and the features of each audio frame are spliced in chronological order to obtain the corresponding audio frame features. training features.

S104: Input the training feature into the initial model to obtain the corresponding training sound quality detection result.

S105: Calculate a loss value using the training sound quality detection result and the average opinion score label, and use the loss value to adjust the model parameters of the initial model.

The above two steps are comprehensively explained.

The initial model refers to a model that has not been fully trained. After sufficient training and parameter adjustment, the adjusted initial model can be used as a sound quality detection model. This embodiment does not limit the specific form and category of the initial model, for example, it may be a convolutional neural network model, or may be a combination of a convolutional neural network and a recurrent neural network. After the processing model processes the training features, a training sound quality detection result obtained by performing sound quality detection on the training audio based on the current learning and parameter adjustment conditions can be obtained.

Without sufficient training, there is a certain gap between the training sound quality detection results obtained by the initial model and the truly correct results (that is, MOS tags). By calculating the loss value and adjusting the model parameters of the initial model based on the loss value, the model can learn How to correctly perform sound quality detection and give correct sound quality detection results.

Specifically, in one embodiment, the initial model includes a convolutional neural network, a long short-term memory network, a fully connected layer, and an average pooling layer. Among them, the convolutional neural network is used to perform convolution calculation on the input training features to extract effective audio local features. Long Short-Term Memory (LSTM), specifically a Bi-directional Long-Short Term Memory (BLSTM), which is used to extract the temporal relationship between audio local features, learn the frames before and after correlation between. The fully connected layer is used to predict the sound quality detection result corresponding to each frame in units of frames. The average pooling layer is used to synthesize the sound quality detection results of each frame to obtain the final training sound quality detection results.

Correspondingly, the process of inputting the training features into the initial model and obtaining the corresponding training sound quality detection results may include:

Step 31: Input the training features into the convolutional neural network to obtain training intermediate features.

Step 32: Input the training intermediate features into the long short-term memory network to obtain the training initial detection results.

Step 33: Input the training initial detection result into the fully connected layer to obtain the training intermediate detection result.

Step 34: Input the training intermediate detection result into the average pooling layer to obtain the training sound quality detection result.

Among them, the training intermediate feature is the feature obtained after convolution calculation, and the initial detection result of training is the data obtained after the long short-term memory network extracts the time series relationship. The training intermediate detection result is the sound quality detection result corresponding to each frame in the training feature.

It should be noted that the specific calculation method of the loss value is not limited, and can be calculated according to the initial model type, the expression of the training sound quality detection results, the training focus direction, etc., for example, the square loss function, the exponential loss function, and the cross entropy loss can be used. function etc. In a specific implementation, the loss value may be calculated by synthesizing the training sound quality detection results corresponding to each initial training audio in a training batch. In this case, when obtaining the initial training audio and corresponding MOS labels, a batch of multiple initial training audio and average opinion score labels can be obtained from the training dataset according to the preset batch size. The preset batch size refers to the number of initial training audios obtained in each training batch. Since the frequency of parameter adjustment is the same as the frequency of loss value generation, in this embodiment, after each batch of initial training audio is processed, a loss value calculation and parameter adjustment are performed.

Correspondingly, the process of calculating the loss value using the training sound quality detection result and the mean opinion score label may include:

Step 41: When the training sound quality detection results corresponding to all the initial training audios corresponding to a batch are obtained, the loss value is obtained by using the training sound quality detection results, the average opinion score label and the training intermediate detection results in the batch.

In this embodiment, the loss value calculation is performed by combining all the training sound quality detection results and the average opinion score label in a batch, and the parameters can be adjusted by combining the training conditions of a whole batch. In addition, the loss value is calculated using the MOS tag and the training intermediate detection result, which can reflect the sound quality evaluation of each frame in the loss value.

Specifically, according to

get the loss value.

为训练音质检测结果，

为训练特征中的第t帧在训练中间检测结果中对应的值，α为预设权重，其具体大小不做限定，例如可以为1。Among them, loss is the loss value, S is the preset batch size, T _S is the number of frames corresponding to the training feature, that is, the audio frame divided when the training feature is generated, M _S is the average opinion score label,

In order to train the sound quality detection results,

is the value corresponding to the t-th frame in the training feature in the intermediate detection result of the training, α is a preset weight, and its specific size is not limited, for example, it may be 1.

S106: When it is detected that the training completion condition is satisfied, the adjusted initial model is determined as the sound quality detection model.

The above steps are executed cyclically, and after each training round is executed and parameters are adjusted, it is judged whether the training completion condition is met. The training completion condition refers to a condition indicating that the initial model has been sufficiently trained, which may specifically be a condition for limiting the training process or a condition for limiting the performance of the initial model. For example, it can be a condition of the number of training rounds, or it can be a condition of a loss value interval. Exemplarily, it may be determined whether the loss value is less than the preset threshold, and if it is less than the preset threshold, it means that it is determined that the training completion condition is satisfied. The specific size of the preset threshold is not limited, and can be set as required.

The sound quality detection model training method provided in the embodiment of the present application is applied, and the mean opinion score (meanopinion score, MOS) is used as the initial training audio and the label of the training audio. Mean Opinion Scores were assessed by a large audience for the quality of audio of sentences read aloud by a male or female speaker, transmitted over a communication circuit. Listeners rated each sentence on the following scale: (1) very poor (2) poor (3) fair (4) good (5) very good, MOS is an arithmetic measure of individual scores for all listeners, ranging from 1 (worst) to 5 (best). The resulting MOS labels can accurately characterize the quality of the sentence audio in the initial training audio. Through speech endpoint detection, the part of non-human voice audio can be used as interference and filtered, and only the part of human voice can be retained as training audio. After the training features are obtained through feature extraction, the initial model is used for quality detection, and the loss value is calculated using the obtained training sound quality detection results and the average opinion score label. The obtained training sound quality detection results are as close as possible to the MOS labels. After the training is completed, the adjusted initial model can be determined as a sound quality detection model. The resulting sound quality detection model can accurately evaluate the quality of the audio under test without having pure clean audio.

Based on the above embodiment, after the sound quality detection model is obtained, the sound quality detection can be performed by using the initial to-be-tested audio that does not have a MOS tag. Please refer to FIG. 4. FIG. 4 is a flowchart of a sound quality detection method provided by an embodiment of the present application, which specifically includes the following steps:

S201: Obtain the initial audio to be tested.

S202: Perform non-human voice filtering processing based on voice endpoint detection on the initial audio to be tested to obtain audio to be tested.

S203: Perform feature extraction processing on the audio to be tested to obtain features to be tested.

S204: Input the feature to be tested into the sound quality detection model to obtain a sound quality detection result corresponding to the audio to be initially tested.

Among them, the sound quality detection model is obtained by the above-mentioned training method of the sound quality detection model, and the non-human voice filtering and feature extraction processing are the same as the training process of the sound quality detection model. Specifically, in one embodiment, the sound quality detection model includes a convolutional neural network, a long short-term memory network, a fully connected layer and an average pooling layer.

Correspondingly, the feature to be tested is input into the sound quality detection model, and the sound quality detection result corresponding to the initial to-be-tested audio is obtained, including:

Step 51: Input the feature to be tested into the convolutional neural network to obtain the intermediate feature to be tested.

Step 52: Input the intermediate features to be tested into the long short-term memory network to obtain an initial detection result.

Step 53: Input the initial detection result into the fully connected layer to obtain the intermediate detection result.

Step 54: Input the intermediate detection result into the average pooling layer to obtain the sound quality detection result.

Wherein, for the process of step 51 to step 54, reference may be made to the process of step 31 to step 34, the difference is that the data to be processed are different.

Further, please refer to FIG. 5 , which is a flowchart of another sound quality detection method provided by an embodiment of the present application. After the audio signal (that is, the initial audio to be tested) is acquired, the VAD algorithm is used to detect the human voice, and the signal of the human voice (that is, the audio to be tested) is output. The VAD algorithm may specifically be the webrtc-vad algorithm. Vocal detection filters out the silent and non-vocal parts of the audio signal. It is then resampled to 48kHz. Extract the audio features (that is, the features to be tested) in the form of Mel spectrum, and input the audio features into the sound quality detection model. A Blackman-Harris window is used for audio feature extraction, and the frame shift size is 10.7ms.

The audio detection model in this application includes a 3-layer CNN network, a 2-layer BLSTM network, a fully connected layer and an average pooling layer. The convolution layer in the CNN network is a 2D convolution layer, the convolution kernel is 3*3, and the output filter lengths of the three convolution layers are 16, 32, and 64 in turn. After each layer of CNN, normalization is performed and a ReLU activation function is used. Through 3-layer CNN, audio local features can be effectively learned. Then it is sent to the 2-layer bidirectional LSTM network, and its hidden units are all set to 256. The main purpose is to extract the temporal relationship of local features and learn the correlation between the front and rear frames. After 2 layers of bidirectional LSTM cascade, it is sent to the fully connected layer to predict the sound quality score of each frame level (ie, the intermediate detection result), and finally through the average pooling layer, the final objective evaluation score corresponding to the initial audio to be tested is output. value (that is, the sound quality detection result).

The computer-readable storage medium provided by the embodiments of the present application will be introduced below. The computer-readable storage medium described below and the voice quality detection model training method described above may refer to each other correspondingly.

The present application also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned training method for a sound quality detection model are implemented.

The computer-readable storage medium may include: a USB flash drive, a removable hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc., which can store program codes. medium.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments may be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.

Those skilled in the art may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the hardware and software In the above description, the components and steps of each example have been generally described according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different approaches to implement the described functionality for each particular application, but such implementations should not be considered beyond the scope of this application.

The steps of a method or algorithm described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.

Finally, it should also be noted that, in this context, relationships such as first and second, etc., are used only to distinguish one entity or operation from another, and do not necessarily require or imply these entities or that there is any such actual relationship or sequence between operations. Furthermore, the terms including, comprising or any other variation are intended to cover non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or Yes also includes elements inherent to such a process, method, article or apparatus.

The principles and implementations of the present application are described herein by using specific examples. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application. There will be changes in the specific implementation and application scope. To sum up, the content of this specification should not be construed as a limitation to the application.

Claims (13)

1. a sound quality detection model training method, is characterized in that, comprises: Obtaining the initial training audio and the corresponding average opinion score label; the average opinion score label is used to represent the average sound quality evaluation parameter obtained after a plurality of evaluation objects perform sound quality evaluation on the initial training audio; Carrying out the non-human voice filtering process based on voice endpoint detection to the initial training audio to obtain the training audio; Perform feature extraction processing on the training audio to obtain training features; Input the training feature into the initial model to obtain the corresponding training sound quality detection result; Calculate a loss value using the training sound quality detection result and the average opinion score label, and use the loss value to adjust the model parameters of the initial model; When it is detected that the training completion condition is satisfied, the adjusted initial model is determined as a sound quality detection model. 2. The sound quality detection model training method according to claim 1, wherein the acquisition process of the average opinion score label comprises: playing the initial training audio to each of the evaluation objects; receiving initial sound quality data obtained after each of the evaluation objects performs sound quality evaluation on the initial training audio; The mean opinion score label is generated using each of the initial voice quality data. 3. The method for training a sound quality detection model according to claim 2, wherein the generating the average opinion score label using each of the initial sound quality data, comprising: Perform averaging processing on each of the initial sound quality data to obtain a first score label; Inputting the initial training audio into an audio defect detection model to obtain a defect detection result; wherein the audio defect detection model is used to detect audio defects that can affect auditory experience; generating a second score label based on the defect detection result; The average opinion score label is generated using the first score label and the second score label. 4. The method for training a sound quality detection model according to claim 1, wherein the described training audio is subjected to feature extraction processing to obtain training features, comprising: Based on the maximum sampling rate perceivable by the human ear, the training audio is resampled to obtain intermediate data; Performing sliding window framing based on a preset window length on the intermediate data to obtain a plurality of audio frames; Feature extraction processing is performed on each of the audio frames to obtain the training features. 5. sound quality detection model training method according to claim 1, is characterized in that, described initial training audio frequency is carried out the non-human voice filtering processing based on voice endpoint detection, obtains training audio frequency, comprises: The voice endpoint detection is carried out to the initial training audio, and the voice endpoint time is obtained; According to the voice endpoint time, the initial training audio is segmented to obtain a plurality of audio segments, and the non-human voice frequency range in the audio segment is removed to obtain the human voice frequency range; The training audio is obtained by splicing the human voice frequency bands. 6. The sound quality detection model training method according to claim 1, wherein the initial model comprises a convolutional neural network, a long short-term memory network, a fully connected layer and an average pooling layer; The described training features are input into the initial model to obtain corresponding training sound quality detection results, including: Inputting the training features into the convolutional neural network to obtain training intermediate features; Inputting the training intermediate feature into the long short-term memory network to obtain the training initial detection result; Inputting the training initial detection result into the fully connected layer to obtain the training intermediate detection result; The training intermediate detection result is input into the average pooling layer to obtain the training sound quality detection result. 7. The method for training a sound quality detection model according to claim 6, wherein the acquisition of the initial training audio and the corresponding average opinion score label comprises: According to a preset batch size, obtain a batch of a plurality of the initial training audio and the average opinion score label from the training data set; Correspondingly, calculating the loss value using the training sound quality detection result and the average opinion score label includes: When the training sound quality detection results corresponding to all the initial training audios corresponding to a batch are obtained, the loss value is obtained by using the training sound quality detection results, the average opinion score label and the training intermediate detection results in the batch . 8. The method for training a sound quality detection model according to claim 7, wherein the loss value is obtained by using the training sound quality detection result, the average opinion score label and the training intermediate detection result in the batch, comprising: : according to

obtain the loss value; Wherein, the loss is the loss value, the S is the preset batch size, the T _S is the number of frames corresponding to the training feature, the M _S is the average opinion score label, and the stated

For the training sound quality detection result, the

is the value corresponding to the t-th frame in the training feature in the intermediate detection result of the training, and the α is a preset weight. 9. The method for training a sound quality detection model according to claim 1, wherein the detection meets the training completion condition, comprising: judging whether the loss value is less than a preset threshold; If so, it is determined that the training completion condition is satisfied. 10. A sound quality detection method, characterized in that, comprising: Get the initial audio to be tested; Carrying out non-human voice filtering processing based on voice endpoint detection to the initial audio to be tested to obtain audio to be tested; Perform feature extraction processing on the audio to be tested to obtain features to be tested; Input the feature to be tested into a sound quality detection model to obtain a sound quality detection result corresponding to the initial audio to be tested; wherein, the sound quality detection model is obtained by using the sound quality detection model training method described in any one of claims 1 to 7 . 11. The sound quality detection method according to claim 10, wherein the sound quality detection model comprises a convolutional neural network, a long short-term memory network, a fully connected layer and an average pooling layer; The inputting the feature to be measured into the sound quality detection model to obtain the sound quality detection result corresponding to the initial to-be-measured audio, including: Inputting the feature to be tested into the convolutional neural network to obtain an intermediate feature to be tested; Inputting the intermediate feature to be tested into the long short-term memory network to obtain an initial detection result; Inputting the initial detection result into the fully connected layer to obtain an intermediate detection result; The intermediate detection result is input into the average pooling layer to obtain the sound quality detection result. 12. An electronic device, comprising a memory and a processor, wherein: the memory for storing computer programs; The processor is used to execute the computer program to realize the sound quality detection model training method according to any one of claims 1 to 9, and/or, the sound quality according to any one of claims 10 to 11 Detection method. 13. A computer-readable storage medium, characterized in that it is used for saving a computer program, wherein, when the computer program is executed by a processor, the method for training a sound quality detection model according to any one of claims 1 to 9 is realized, and/or, the sound quality detection method according to any one of claims 10 to 11.

Images