CN118645101A - Intelligent sound box control system - Google Patents
Intelligent sound box control system Download PDFInfo
- Publication number
- CN118645101A CN118645101A CN202411096568.8A CN202411096568A CN118645101A CN 118645101 A CN118645101 A CN 118645101A CN 202411096568 A CN202411096568 A CN 202411096568A CN 118645101 A CN118645101 A CN 118645101A
- Authority
- CN
- China
- Prior art keywords
- sound box
- resource
- parameters
- performance
- representing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000004044 response Effects 0.000 claims abstract description 76
- 238000012545 processing Methods 0.000 claims abstract description 63
- 238000004458 analytical method Methods 0.000 claims abstract description 32
- 230000001629 suppression Effects 0.000 claims abstract description 27
- 230000033228 biological regulation Effects 0.000 claims abstract description 24
- 230000015654 memory Effects 0.000 claims abstract description 19
- 238000012544 monitoring process Methods 0.000 claims abstract description 5
- 238000007405 data analysis Methods 0.000 claims abstract description 4
- 238000005457 optimization Methods 0.000 claims description 34
- 230000035945 sensitivity Effects 0.000 claims description 26
- 230000005236 sound signal Effects 0.000 claims description 24
- 238000013468 resource allocation Methods 0.000 claims description 21
- 230000007613 environmental effect Effects 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 9
- 238000000034 method Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 230000002708 enhancing effect Effects 0.000 claims description 7
- 238000011156 evaluation Methods 0.000 claims description 7
- 230000004927 fusion Effects 0.000 claims description 7
- 238000007726 management method Methods 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 230000002688 persistence Effects 0.000 claims description 4
- 238000001228 spectrum Methods 0.000 claims description 4
- 230000003213 activating effect Effects 0.000 claims description 3
- 230000001105 regulatory effect Effects 0.000 claims description 3
- 238000010183 spectrum analysis Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 7
- 238000005265 energy consumption Methods 0.000 abstract description 3
- 230000003993 interaction Effects 0.000 abstract description 3
- 238000004364 calculation method Methods 0.000 description 49
- 238000005516 engineering process Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000007619 statistical method Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000036632 reaction speed Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Tone Control, Compression And Expansion, Limiting Amplitude (AREA)
Abstract
The invention relates to the technical field of voice control, in particular to an intelligent sound box control system which comprises a voice data analysis module, a dynamic resource management module, a noise suppression adjustment module and an intelligent response speed regulation module. According to the invention, through accurate analysis of voice input of the loudspeaker box, recognition precision and execution effect of voice command are improved, user interaction experience is remarkably improved, processing capacity and data efficiency of complex commands are enhanced by utilizing an autoregressive integral moving average model and a long-short-term memory network, dynamic resource management is implemented, CPU and memory configuration is automatically adjusted according to command prediction, response speed and resource utilization of equipment are optimized, system processing capacity is improved, real-time noise suppression ensures that instructions can be accurately captured in a multi-noise environment, and in addition, through continuous monitoring and optimizing of response speed and load of the intelligent loudspeaker box, operation efficiency is improved, and energy consumption and processing load are reduced.
Description
Technical Field
The invention relates to the technical field of voice control, in particular to an intelligent sound box control system.
Background
The field of speech control technology, which involves the manipulation and control of devices through sound and speech commands, uses a speech recognition system to parse a user's spoken commands, convert them into executable commands to control various devices and services, includes speech-to-text conversion, command recognition, natural Language Processing (NLP), and development of machine learning algorithms, to improve recognition accuracy and the ability to understand complex commands. The voice control technology is widely applied to home automation, vehicle-mounted systems, smart phone application and auxiliary technologies, and has the core advantages of providing a non-contact operation mode, enhancing user experience and enabling equipment operation to be more visual and convenient.
Wherein the intelligent sound box control system is a system for operating the intelligent sound box by using voice control technology. The intelligent sound box is equipment provided with a microphone, a loudspeaker and Internet connection, can respond to voice commands of a user to execute various functions, such as playing music, setting an alarm clock, providing weather forecast, controlling intelligent household equipment and the like, and can communicate with the equipment only through voice interaction without manual operation by the intelligent sound box control system, so that daily life is more convenient and intelligent, and the system is mainly used for simplifying equipment management in a home or office environment through a natural language interface and improving the automation level of life.
Although the existing intelligent sound box control system is widely applied to various sound box devices, the existing intelligent sound box control system still has defects in terms of voice recognition accuracy, resource management efficiency and environmental noise processing, particularly in complex or noisy environments, voice commands of users are difficult to accurately capture and process due to environmental noise in the prior art, and the traditional voice recognition system lacks flexibility in resource allocation, so that the traditional voice recognition system runs poorly on devices with limited resources, and user experience is affected. The optimization of the response speed also fails to meet the requirement of real-time update, which limits the utility of the voice control technology in emergency or application scenes requiring quick response, not only affects the operation convenience of users, but also limits the application and popularization of the voice technology in wider scenes.
Disclosure of Invention
The invention aims to solve the defects in the prior art and provides an intelligent sound box control system.
In order to achieve the above purpose, the present invention adopts the following technical scheme: an intelligent sound box control system includes:
The voice data analysis module is used for analyzing input data through an autoregressive integral moving average model based on voice input data of the loudspeaker box to obtain a voice mode analysis result, and a command prediction result after refinement is obtained by utilizing a long-short-term memory network to refine prediction of a command mode;
the dynamic resource management module dynamically adjusts the resources controlled by the intelligent sound box according to the refined command prediction result, wherein the resources comprise CPU allocation amount and memory configuration, resource optimization is executed, the sound box resource optimization state is obtained, and the operation parameters are adjusted through the sound box resource optimization state, so that the optimized resource configuration is obtained;
The noise suppression adjustment module captures environmental audio data based on the optimized resource configuration, analyzes and updates the state of the audio signal through a Kalman filter to obtain an audio state adjustment result, suppresses environmental noise in real time by utilizing the audio state adjustment result, captures a voice command controlled by the intelligent sound box, and obtains an optimized voice capturing result;
And the intelligent response speed regulation and control module monitors response time and processing load of the intelligent sound box based on the optimized voice capturing result, generates a sound box performance log, analyzes response efficiency and load conditions according to the sound box performance log, optimizes response speed and obtains regulated sound box performance regulation and control response configuration.
As a further scheme of the invention, the step of obtaining the command prediction result after refinement specifically comprises the following steps:
based on voice input data of a loudspeaker box, performing signal processing by applying frequency spectrum analysis, extracting fundamental frequency and sound spectrum characteristics, and adopting the formula:
；
Calculating fundamental frequency characteristics, generating signal characteristic analysis results, wherein, Representing the result of the fundamental frequency analysis,Is the i-th frequency component and,Is the corresponding amplitude of the light beam,Is the number of sample points and,Is a persistence index;
and utilizing the signal characteristic analysis result, adopting a long-short-period memory network to learn and predict the command mode, and adopting the formula:
；
Enhancing the sensitivity of the model to input variation, calculating command pattern probability, generating preliminary command pattern prediction results, wherein, Is the predicted probability of the command pattern,Is a weight parameter that is used to determine the weight of the object,Is an output function of the LSTM cell,Is an input characteristic of the character,Is the number of features that are to be used,Is the coefficient of variation;
and using the preliminary command mode prediction result and combining the context information of the current environment to adopt the formula:
；
Calculating and obtaining a command prediction result after refinement, wherein, Is the result of the prediction of the command,Is the probability of the jth predicted command,Is the number of commands to be referred to,Is the standard deviation.
As a further scheme of the invention, the acquisition steps of the sound box resource optimization state are specifically as follows:
extracting the resource demand sensitivity parameters of the command from the refined command prediction result, calculating the demand of the command on the resource, and adopting the formula:
；
A refinement command resource requirement is generated, wherein, Representing the need for resources,Representing the resource sensitivity of the command prediction,Is an adjustment coefficient;
according to the resource requirement of the refinement command, the CPU and memory configuration of the intelligent sound box is adjusted, and the formula is adopted:
；
generating an updated resource allocation, wherein, Indicating the updated resource allocation situation,Representing the need for resources,Is an adjustment parameter;
based on the updated resource allocation condition, monitoring and recording the running state of the resource after optimization, and adopting the formula:
；
Generating a sound box resource optimization state, wherein, Is the optimized state of sound box resources,Is the case for the updated resource allocation,Is a performance evaluation parameter.
As a further aspect of the present invention, the step of obtaining the optimized resource configuration specifically includes:
According to the optimized state of the sound box resource, adopting the formula:
；
generating an optimized state parameter after analysis, wherein, Representing the parameters of the optimized state after analysis,Is the gain factor of the gain factor,Is a baseline correction parameter that is used to correct the baseline,Is the optimized state of sound box resource;
according to the analyzed optimized state parameters, calculating the required CPU and memory resources, and adopting the formula:
；
Generating an estimated resource parameter, wherein, Representing the parameters of the resource after the evaluation,Is an adjustment factor that is used to adjust the position of the device,Representing the optimized state parameters after analysis;
and adjusting the operation parameters by using the evaluated resource parameters, optimizing the sound control performance, and adopting the formula:
；
Generating an optimized resource configuration, wherein, Is the configuration of the resource after the optimization,Is a weight parameter that is used to determine the weight of the object,The proportion is adjusted to be a certain value,AndRepresenting the assessed resource parameters and the current resource configuration, respectively.
As a further aspect of the present invention, the step of obtaining the audio status adjustment result specifically includes:
based on the optimized resource configuration, activating audio capture, collecting environmental audio data, and adopting the formula:
；
a captured audio data parameter is generated, wherein, Representing the captured audio data and,AndIs the gain and sensitivity adjustment coefficient,Is the ambient audio intensity;
inputting the captured audio data parameters into a Kalman filter, analyzing and updating the state of an audio signal, adjusting the smoothness and response speed of the signal, and adopting the formula:
；
a filtered audio signal is generated, wherein, Representing the audio signal after the filtering,And gamma is the sensitivity and threshold that regulates signal processing,Representing captured audio data;
based on the filtered audio signal, updating the audio state by adopting the formula:
；
an audio state adjustment result is generated, wherein, Is the result of the audio state adjustment,AndThe update rate and adjustment range of the audio state are adjusted,Representing the filtered audio signal.
As a further aspect of the present invention, the step of obtaining the optimized voice capturing result specifically includes:
According to the audio state adjustment result, real-time noise suppression is executed, audio capturing in the environment is optimized, and the formula is adopted:
；
generating noise-suppressed audio parameters, wherein, In order to optimize the noise suppression parameters after the optimization,Noise suppression gain, stability coefficient and sensitivity adjustment parameters,Representing an audio state adjustment result;
And capturing and primarily processing the voice command controlled by the intelligent sound box by utilizing the audio parameters after noise suppression, wherein the voice command is represented by the formula:
；
generating the voice parameters after preliminary processing, wherein, For the initially processed voice command parameters,AndIs the processing gain and weight parameters that are used,In order to optimize the noise suppression parameters after the optimization,Is the original voice data in the environment;
Updating the voice recognition configuration of the intelligent sound box according to the primarily processed voice parameters, and adopting the formula:
；
generating an optimized speech capture result, wherein, In order to optimize the result of the speech capture,The update coefficient is represented by a number of coefficients,Is the voice instruction parameter after preliminary processing.
As a further scheme of the invention, the acquisition steps of the sound box performance log specifically comprise:
And optimizing the representation of response time by using the optimized voice capturing result through the formula:
；
a response time performance index is generated, wherein, AndRepresenting the gain factor, the base bias and the adjustment factor,Representing the performance index of the response time,Representing the optimized voice capturing result;
and combining the response time performance index and the processing load, adopting the formula:
；
Generating a performance log parameter, wherein, As a function of the performance log parameters,AndIs a weight parameter that is used to determine the weight of the object,Representing the performance index of the response time,Representing a processing load;
using the performance log parameters, by configuring the sound box parameters and recording key performance data, the formula is adopted:
；
generating a sound box performance log, wherein, Is the coefficient of fusion and is used for the fusion,Is a log of the previous performance of the device,Is the performance log of the sound box,Is a performance log parameter.
As a further scheme of the invention, the acquisition steps of the adjusted sound box performance regulation response configuration are specifically as follows:
extracting key performance indexes from the sound box performance log, wherein the key performance indexes comprise response efficiency and processing load, calculating preliminary performance data, and adopting the formula:
；
generating a performance analysis result, wherein, Representing performance data extracted from the log, 、The weight coefficient of the weight of the sample,Representing the processing load of the device,Representing response efficiency;
Based on the performance analysis result, processing parameters of the sound box are adjusted, and the response speed of the equipment is optimized, through the formula:
；
an optimized processing configuration is generated, wherein, In order to optimize the configuration of the process,AndIs the adjustment parameter of the device, which is used for adjusting the parameters,Indicating that the processing parameters of the sound box are adjusted,Representing performance data extracted from the log;
Updating the performance regulation configuration of the sound box by using the optimized processing configuration, and applying the formula:
；
Obtaining the adjusted sound box performance regulation response configuration, wherein, Is the update coefficient of the coefficient,Is the prior sound box performance regulation configuration,In order to optimize the configuration of the process,And representing the adjusted sound box performance regulation response configuration.
Compared with the prior art, the invention has the advantages and positive effects that:
according to the invention, through careful analysis and processing of voice input of the loudspeaker box, recognition precision and execution precision of voice commands are improved, user interaction experience is remarkably improved, an autoregressive integral moving average model and a long-short-term memory network are adopted to analyze voice modes, data processing efficiency is optimized, understanding capacity of complex commands is improved, dynamic resource management is implemented, CPU and memory configuration is adjusted according to prediction results, equipment can automatically optimize resource use under different requirements, and overall response speed and processing capacity of a system are improved. The voice capturing quality is further improved through real-time noise suppression, the user instruction can be accurately identified under the noisy environment, the response speed and the processing load of the intelligent sound box are monitored and optimized, the equipment is kept to operate at high efficiency, meanwhile, the energy consumption can be saved, and the unnecessary processing load is reduced.
Drawings
FIG. 1 is a system flow diagram of the present invention;
FIG. 2 is a flow chart of the refined command prediction result in the present invention;
FIG. 3 is a flow chart of the invention in the optimized state of the sound box resources;
FIG. 4 is a flow chart of an optimized resource configuration in the present invention;
FIG. 5 is a flow chart of the audio status adjustment result according to the present invention;
FIG. 6 is a flow chart of the optimized speech capture result of the present invention;
FIG. 7 is a flow chart of the sound box performance log of the present invention;
FIG. 8 is a flow chart of an adjusted loudspeaker performance tuning response configuration in accordance with the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In the description of the present invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like indicate orientations or positional relationships based on the orientation or positional relationships shown in the drawings, merely to facilitate describing the present invention and simplify the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present invention. Furthermore, in the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
Examples: referring to fig. 1, an intelligent sound box control system includes:
The voice data analysis module is used for analyzing input data through an autoregressive integral moving average model based on voice input data of the loudspeaker box to obtain a voice mode analysis result, and a command prediction result after refinement is obtained by utilizing a long-short-term memory network to refine prediction of a command mode;
The dynamic resource management module dynamically adjusts the resources controlled by the intelligent sound box according to the refined command prediction result, wherein the resources comprise CPU allocation amount and memory configuration, resource optimization is executed, the sound box resource optimization state is obtained, and the operation parameters are adjusted through the sound box resource optimization state, so that the optimized resource configuration is obtained;
The noise suppression adjustment module captures environmental audio data based on the optimized resource configuration, analyzes and updates the state of the audio signal through a Kalman filter to obtain an audio state adjustment result, suppresses environmental noise in real time by utilizing the audio state adjustment result, captures a voice command controlled by the intelligent sound box, and obtains an optimized voice capturing result;
The intelligent response speed regulation and control module monitors response time and processing load of the intelligent sound box based on the optimized voice capturing result, generates a sound box performance log, analyzes response efficiency and load conditions according to the sound box performance log, optimizes response speed, and obtains regulated sound box performance regulation and control response configuration.
The voice mode analysis result comprises tone distribution, voice frequency spectrum and speech speed interval, the refined command prediction result comprises command intention type, parameter setting range and execution priority, the sound box resource optimization state comprises processor load, memory occupancy rate and energy consumption level, the optimized resource configuration comprises resource allocation scheme, storage optimization level and processing capacity improvement, the audio state adjustment result comprises noise elimination effect, voice enhancement level and signal stability, the optimized voice capture result comprises recognition precision improvement, reaction speed optimization and false recognition rate reduction, the sound box performance log comprises performance peak value record, response abnormality record and resource use history, and the optimized sound box performance regulation response configuration comprises time response optimization, load balance strategy and performance self-adaptive adjustment.
Referring to fig. 2, the steps for obtaining the refined command prediction result specifically include:
based on voice input data of a loudspeaker box, performing signal processing by applying frequency spectrum analysis, extracting fundamental frequency and sound spectrum characteristics, and adopting the formula:
；
Calculating fundamental frequency characteristics, generating signal characteristic analysis results, wherein, Representing the result of the fundamental frequency analysis,Is the i-th frequency component and,Is the corresponding amplitude of the light beam,Is the number of sample points and,Is a persistence index;
And (3) utilizing a signal characteristic analysis result, adopting a long-term and short-term memory network to learn and predict a command mode, and adopting the following formula:
；
Enhancing the sensitivity of the model to input variation, calculating command pattern probability, generating preliminary command pattern prediction results, wherein, Is the predicted probability of the command pattern,Is a weight parameter that is used to determine the weight of the object,Is an output function of the LSTM cell,Is an input characteristic of the character,Is the number of features that are to be used,Is the coefficient of variation;
using the preliminary command mode prediction result, in combination with the context information of the current environment, adopting the formula:
；
Calculating and obtaining a command prediction result after refinement, wherein, Is the result of the prediction of the command,Is the probability of the jth predicted command,Is the number of commands to be referred to,Is the standard deviation.
The formula of the fundamental frequency analysis result:
；
: the i-th frequency component is typically obtained by converting a time domain signal into a frequency domain signal by fourier transform;
: corresponding frequency Is obtained by fourier transform results;
: the persistence index of the signal can be obtained by analyzing the time length of the signal at a certain frequency;
: the number of sampling points represents the total frequency component in the signal;
assume that the signal has three frequency components Hz, corresponding amplitudePersistence index of each componentTotal sampling point；
Calculation process：
；
；
This isThe value represents the fundamental component of the weighted average of the signal, a quantized representation of the signal characteristics.
Prediction probability formula of command mode:
；
: the weight parameters reflect the importance of each feature and are generally determined through statistical analysis of training data;
: the output function of LSTM cells, dependent on input characteristics And parameters of the network;
: input features, which are preprocessed signal features such as MFCCs, etc.;
: coefficient of variation, characteristic of Variability in the training set can be obtained by statistical methods;
: feature quantity;
Assuming three features, weights ，Coefficient of variationFeature number；
Calculation of：
；
；
；
This isThe value represents the probability of the predicted command pattern after the variability is taken into account.
Command prediction result formula:
；
: probability of jth predicted command, from the above Calculated;
: standard deviation, expressed as Is obtained by statistical analysis;
: the number of commands considered;
Hypothesis prediction results Standard deviation ofNumber of commands；
Calculation of：
；
；
；
；
This isThe value represents a measure of stability of the final command prediction after the integrated standard deviation adjustment.
Referring to fig. 3, the steps for obtaining the optimized state of the sound box resource specifically include:
Extracting the resource demand sensitivity parameters of the command from the refined command prediction result, calculating the demand of the command on the resource, and adopting the formula:
；
A refinement command resource requirement is generated, wherein, Representing the need for resources,Representing the resource sensitivity of the command prediction,Is an adjustment coefficient;
according to the resource requirement of the refinement command, the CPU and memory configuration of the intelligent sound box is adjusted, and the formula is adopted:
；
generating an updated resource allocation, wherein, Indicating the updated resource allocation situation,Representing the need for resources,Is an adjustment parameter;
based on the updated resource allocation condition, monitoring and recording the running state of the resource after optimizing, and adopting the formula:
；
Generating a sound box resource optimization state, wherein, Is the optimized state of sound box resources,Is the case for the updated resource allocation,Is a performance evaluation parameter.
The resource demand formula:
；
: assuming resource sensitivity parameters for command prediction, which are model predictors obtained by analyzing user command type and frequency, e.g ；
Resource demand adjustment coefficients, provided as fixed values, e.g；
Calculation of：
；
Calculation of：
；
Substituting the above result into：
；
In this example of the calculation of the number of the samples,Representing the predicted resource demand strength based on the command.
The updated resource allocation case formula:
；
From the last step, e.g. ；
: Adjusting parameters, provided with；
Calculation of：
；
Calculating denominator:
；
Substituting the above result into ：
；
In this example of the calculation of the number of the samples,And (3) representing the optimized resource allocation state, wherein the resource allocation algorithm can be dynamically adjusted to improve the efficiency.
And (3) optimizing a state formula of sound box resources:
；
From the last step, e.g. ；
: Performance evaluation parameters, set；
Calculating a second term:
；
Substituting the above result into ：
；
In this example of the calculation of the number of the samples,The final resource optimization state of the sound box is represented, the overall benefit of resource optimization is reflected, and the actual state after performance optimization is quantized.
Referring to fig. 4, the steps for obtaining the optimized resource configuration specifically include:
according to the optimized state of sound box resources, the formula is adopted:
；
generating an optimized state parameter after analysis, wherein, Representing the parameters of the optimized state after analysis,Is the gain factor of the gain factor,Is a baseline correction parameter that is used to correct the baseline,Is the optimized state of sound box resource;
according to the analyzed optimized state parameters, calculating the required CPU and memory resources, and adopting the formula:
；
Generating an estimated resource parameter, wherein, Representing the parameters of the resource after the evaluation,Is an adjustment factor that is used to adjust the position of the device,Representing the optimized state parameters after analysis;
And adjusting the operation parameters by using the evaluated resource parameters, optimizing the sound control performance, and adopting the formula:
；
Generating an optimized resource configuration, wherein, Is the configuration of the resource after the optimization,Is a weight parameter that is used to determine the weight of the object,The proportion is adjusted to be a certain value,AndRepresenting the assessed resource parameters and the current resource configuration, respectively.
The analyzed optimized state parameter formula:
；
: the gain coefficient is assumed to be 2 and is used for enhancing the feedback effect of the state;
: baseline correction parameters, assuming a value of 1, for equilibrium state response;
: the sound box resource optimizing state assumes a value of 3;
Calculation of ：
；
Calculating denominator：
；
Substituting the above result intoThe formula:
；
In this example of the calculation of the number of the samples, Representing the optimized state after analysis, indicating that the optimized state has been effectively amplified and adjusted.
The estimated resource parameter formula:
；
: the adjustment factor, assuming a value of 1.5, is used to optimize the sensitivity of the resource allocation;
: adjusting the factor, and assuming the value to be 0.5;
: the output obtained from the previous step was 4.5;
Calculation of ：
；
Calculating denominator：
；
Substituting the above result intoThe formula:
；
In this example of the calculation of the number of the samples, Representing the calculated resource demand, reflecting the high demand state for the resource allocation.
The optimized resource allocation formula:
；
: the weight parameter, assume the value is 0.7, control the new configuration and balance of the present configuration;
: the adjustment proportion of the current configuration is assumed to be 0.3;
: the output obtained from the previous step was 28.0;
: the current resource allocation state, assumed to be 10;
Calculation of ：
；
Calculation of：
；
Substituting the above result intoThe formula:
；
In this example of the calculation of the number of the samples, And representing the optimized resource configuration, and displaying the actual adjustment result of the resource configuration.
Referring to fig. 5, the step of obtaining the audio status adjustment result specifically includes:
Based on the optimized resource configuration, activating audio capture, collecting environmental audio data, and adopting the formula:
；
a captured audio data parameter is generated, wherein, Representing the captured audio data and,AndIs the gain and sensitivity adjustment coefficient,Is the ambient audio intensity;
inputting the captured audio data parameters into a Kalman filter, analyzing and updating the state of an audio signal, adjusting the smoothness and response speed of the signal, and adopting the formula:
；
a filtered audio signal is generated, wherein, Representing the audio signal after the filtering,And gamma is the sensitivity and threshold that regulates signal processing,Representing captured audio data;
based on the filtered audio signal, updating the audio state using the formula:
；
an audio state adjustment result is generated, wherein, Is the result of the audio state adjustment,AndThe update rate and adjustment range of the audio state are adjusted,Representing the filtered audio signal.
The captured audio data formula:
；
: gain factor for adjusting amplification level of audio data, assuming that ；
: Sensitivity adjustment coefficients for enhancing response to environmental audio intensity, assuming；
: Environmental audio intensity, which is assumed to be measured in a certain environmentA unit;
Calculation of ：
；
Calculation of：
；
Calculation of：
Substituting the above result into：
；
In this example of the calculation of the number of the samples,Representing the processing results of the captured audio data, representing the sensitivity and response to the ambient volume.
The filtered audio signal formula:
；
: filter gain coefficients for adjusting the smoothness of an audio signal, provided that ；
: Filtering the baseline parameters to ensure that the denominator is non-zero, assuming；
: The filter sensitivity adjustment parameters affect the signal response speed, assuming；
: The value obtained from the previous step is 0.7455;
Calculation of ：
；
Calculation of：
；
Calculating denominator:
；
Substituting the above result into ：
；
In this example of the calculation of the number of the samples,Representing the filtered audio signal, shows the effect of the kalman filter in reducing noise and smoothing the signal.
Audio state adjustment result formula:
；
: state update coefficients, adjusting update rate of audio signal, assuming ；
: Amplitude adjustment parameters for periodic adjustment of analog audio signals, provided that；
: The value obtained from the previous step was 0.898;
Calculating sin ：
；
Substituting the above result into：
；
In this example of the calculation of the number of the samples,Representing the final audio state adjustment results, demonstrating the final processing effect of the audio signal achieved by the kalman filter and the state update algorithm.
Referring to fig. 6, the steps for obtaining the optimized voice capturing result specifically include:
According to the audio state adjustment result, real-time noise suppression is executed, audio capturing in the environment is optimized, and the formula is adopted:
；
generating noise-suppressed audio parameters, wherein, In order to optimize the noise suppression parameters after the optimization,Noise suppression gain, stability coefficient and sensitivity adjustment parameters,Representing an audio state adjustment result;
the voice command controlled by the intelligent sound box is captured and primarily processed by utilizing the audio parameters after noise suppression, and the formula is adopted:
；
generating the voice parameters after preliminary processing, wherein, For the initially processed voice command parameters,AndIs the processing gain and weight parameters that are used,In order to optimize the noise suppression parameters after the optimization,Is the original voice data in the environment;
Updating the voice recognition configuration of the intelligent sound box according to the primarily processed voice parameters, and adopting the formula:
；
generating an optimized speech capture result, wherein, In order to optimize the result of the speech capture,The update coefficient is represented by a number of coefficients,Is the processed voice instruction parameter.
The optimized noise suppression parameter formula:
；
(noise suppression gain): set to 0.5 for adjusting the intensity of noise suppression;
(stability factor): setting to 1, ensuring that the denominator is not zero and balancing noise suppression;
(sensitivity adjustment parameter): setting to 0.1, adjusting sensitivity of noise suppression;
(the audio state adjustment result obtained from the previous flow): assume 3;
Calculation of ：
；
Calculation of：
；
Calculating denominator:
；
Substituting the above result into ：
；
In this example of the calculation of the number of the samples,Representing the noise suppressed audio parameters, indicating effective noise suppression gain and stability adjustment.
The voice instruction parameter formula after preliminary treatment:
；
(processing gain): setting the voice data to be 2, and enhancing the definition of the voice data;
(weight parameters): setting the weight of the enhanced environment data to be 0.8;
: obtained from the previous step, a value of 2.59;
(original speech data in the environment): assume 5;
Calculation of ：
；
Calculation of：
；
Substituting the above result into：
；
In this example of the calculation of the number of the samples,Representing the initially processed speech parameters, showing an effective speech enhancement.
The optimized voice capturing result formula:
；
(update coefficients): set to 0.7 for balancing new and old data;
: obtained from the previous step, a value of 4.86;
(previous speech recognition result): assume 3;
Calculation of And：
；
；
Adding the above results to obtain：
；
In this example of the calculation of the number of the samples,And the final optimized voice capturing result is represented, the effective integration of new and old data is displayed, and the continuity and the enhancement effect of voice recognition are ensured.
Referring to fig. 7, the steps for obtaining the performance log of the sound box specifically include:
Optimizing the representation of response time by utilizing the optimized voice capturing result through the formula:
；
a response time performance index is generated, wherein, AndRepresenting the gain factor, the base bias and the adjustment factor,Representing the performance index of the response time,Representing the optimized voice capturing result;
in combination with the response time performance index and the processing load, the formula is adopted:
；
Generating a performance log parameter, wherein, As a function of the performance log parameters,AndIs a weight parameter that is used to determine the weight of the object,Representing the performance index of the response time,Representing a processing load;
Using performance log parameters, by configuring sound box parameters and recording key performance data, adopting the formula:
；
generating a sound box performance log, wherein, Is the coefficient of fusion and is used for the fusion,Is a log of the previous performance of the device,Is the performance log of the sound box,Is a performance log parameter.
Response time performance index formula:
； : a gain coefficient set to 1.5 to enhance the influence of the response time;
: a base bias set to 2.0 to prevent denominator zero;
: an adjustment coefficient set to 0.5 to balance the contribution of response time to performance;
: the optimized speech capture result obtained from the previous processing step, assumed to be 4;
Calculation of ：
；
In calculating denominators：
；
Calculating the complete denominator:
；
Final result Value:
；
This value is Representing the response time performance index, revealing the impact of response time on overall performance.
The performance log parameter formula:
；
: weight coefficient is set to 0.1 for adjusting Contribution of (2);
: an adjustment coefficient set to 1 to enhance Is a function of (1);
: the previous calculation shows that the value is 6;
: the processing load, assumed to be 10;
Calculation of ：
；
Calculation of：
；
Final resultValue:
；
This value is Representing the overall performance log parameters reflecting the overall impact of processing load and response time.
The sound box performance log formula:
；
: the fusion coefficient is set to 0.8 and is used for balancing the influence of new and old data;
: the previous calculation gave a value 24.916;
: the previous performance log, assumed 20;
Calculation of ：
；
Calculation of：
；
Final resultValue:
；
This value is Representing a final enclosure performance log, demonstrating the ability of performance monitoring to continually update and integrate past data.
Referring to fig. 8, the steps for obtaining the adjusted sound box performance regulation response configuration specifically include:
Extracting key performance indexes from the sound box performance log, wherein the key performance indexes comprise response efficiency and processing load, calculating preliminary performance data, and adopting the formula:
；
generating a performance analysis result, wherein, Representing performance data extracted from the log, 、The weight coefficient of the weight of the sample,Representing the processing load of the device,Representing response efficiency;
based on the performance analysis result, processing parameters of the sound box are adjusted, and the response speed of the equipment is optimized, through the formula:
；
an optimized processing configuration is generated, wherein, In order to optimize the configuration of the process,AndIs the adjustment parameter of the device, which is used for adjusting the parameters,Indicating that the processing parameters of the sound box are adjusted,Representing performance data extracted from the log;
Updating the performance regulation configuration of the sound box by using the optimized processing configuration, and applying the formula:
；
Obtaining the adjusted sound box performance regulation response configuration, wherein, Is the update coefficient of the coefficient,Is the prior sound box performance regulation configuration,In order to optimize the configuration of the process,And representing the adjusted sound box performance regulation response configuration.
The performance data formula extracted from the log:
；
The assumed parameters are:
: the weight coefficient enhances the influence of response efficiency;
: a weight coefficient, adjusting sensitivity to a processing load;
: assuming a response efficiency value of 5;
: assume that the processing load is 16;
Calculation of ：
；
Calculation of：
；
Calculating denominator：
；
Calculation of：
；
Final calculation：
；
This indicatesRepresenting the weighted result of the performance data extracted from the log.
The optimized processing configuration formula comprises the following steps:
；
The assumed parameters are:
: adjusting the square effect of the performance data;
: ensuring that the denominator is not zero;
: the flexibility of adjusting log parameters;
: assuming a processing parameter of 3;
Calculation of ：
；
Computational molecules：
；
Calculation of：
；
Calculation of：
；
This indicatesRepresenting based on performance dataIs provided.
The adjusted sound box performance regulation response configuration formula:
；
The assumed parameters are:
: fusing the proportion of new and old configurations;
: assume the previous configuration is 300;
Calculation of ：
；
Calculation of：
；
Calculation of：
；
This indicatesRepresenting the final updated loudspeaker performance tuning response configuration provides a comprehensive configuration based on the previous and new optimization data.
The present invention is not limited to the above embodiments, and any equivalent embodiments which can be changed or modified by the technical disclosure described above can be applied to other fields, but any simple modification, equivalent changes and modification made to the above embodiments according to the technical matter of the present invention will still fall within the scope of the technical disclosure.
Claims (8)
1. An intelligent sound box control system, the system comprising:
The voice data analysis module is used for analyzing input data through an autoregressive integral moving average model based on voice input data of the loudspeaker box to obtain a voice mode analysis result, and a command prediction result after refinement is obtained by utilizing a long-short-term memory network to refine prediction of a command mode;
the dynamic resource management module dynamically adjusts the resources controlled by the intelligent sound box according to the refined command prediction result, wherein the resources comprise CPU allocation amount and memory configuration, resource optimization is executed, the sound box resource optimization state is obtained, and the operation parameters are adjusted through the sound box resource optimization state, so that the optimized resource configuration is obtained;
The noise suppression adjustment module captures environmental audio data based on the optimized resource configuration, analyzes and updates the state of the audio signal through a Kalman filter to obtain an audio state adjustment result, suppresses environmental noise in real time by utilizing the audio state adjustment result, captures a voice command controlled by the intelligent sound box, and obtains an optimized voice capturing result;
And the intelligent response speed regulation and control module monitors response time and processing load of the intelligent sound box based on the optimized voice capturing result, generates a sound box performance log, analyzes response efficiency and load conditions according to the sound box performance log, optimizes response speed and obtains regulated sound box performance regulation and control response configuration.
2. The intelligent sound box control system according to claim 1, wherein the step of obtaining the refined command prediction result specifically comprises:
based on voice input data of a loudspeaker box, performing signal processing by applying frequency spectrum analysis, extracting fundamental frequency and sound spectrum characteristics, and adopting the formula:
；
Calculating fundamental frequency characteristics, generating signal characteristic analysis results, wherein, Representing the result of the fundamental frequency analysis,Is the i-th frequency component and,Is the corresponding amplitude of the light beam,Is the number of sample points and,Is a persistence index;
and utilizing the signal characteristic analysis result, adopting a long-short-period memory network to learn and predict the command mode, and adopting the formula:
；
Enhancing the sensitivity of the model to input variation, calculating command pattern probability, generating preliminary command pattern prediction results, wherein, Is the predicted probability of the command pattern,Is a weight parameter that is used to determine the weight of the object,Is an output function of the LSTM cell,Is an input characteristic of the character,Is the number of features that are to be used,Is the coefficient of variation;
and using the preliminary command mode prediction result and combining the context information of the current environment to adopt the formula:
；
Calculating and obtaining a command prediction result after refinement, wherein, Is the result of the prediction of the command,Is the probability of the jth predicted command,Is the number of commands to be referred to,Is the standard deviation.
3. The intelligent sound box control system according to claim 2, wherein the step of obtaining the sound box resource optimization state specifically comprises:
extracting the resource demand sensitivity parameters of the command from the refined command prediction result, calculating the demand of the command on the resource, and adopting the formula:
；
A refinement command resource requirement is generated, wherein, Representing the need for resources,Representing the resource sensitivity of the command prediction,Is an adjustment coefficient;
according to the resource requirement of the refinement command, the CPU and memory configuration of the intelligent sound box is adjusted, and the formula is adopted:
；
generating an updated resource allocation, wherein, Indicating the updated resource allocation situation,Representing the need for resources,Is an adjustment parameter;
based on the updated resource allocation condition, monitoring and recording the running state of the resource after optimization, and adopting the formula:
；
Generating a sound box resource optimization state, wherein, Is the optimized state of sound box resources,Is the case for the updated resource allocation,Is a performance evaluation parameter.
4. The intelligent sound box control system according to claim 3, wherein the step of obtaining the optimized resource configuration specifically includes:
According to the optimized state of the sound box resource, adopting the formula:
；
generating an optimized state parameter after analysis, wherein, Representing the parameters of the optimized state after analysis,Is the gain factor of the gain factor,Is a baseline correction parameter that is used to correct the baseline,Is the optimized state of sound box resource;
according to the analyzed optimized state parameters, calculating the required CPU and memory resources, and adopting the formula:
；
Generating an estimated resource parameter, wherein, Representing the parameters of the resource after the evaluation,Is an adjustment factor that is used to adjust the position of the device,Representing the optimized state parameters after analysis;
and adjusting the operation parameters by using the evaluated resource parameters, optimizing the sound control performance, and adopting the formula:
；
Generating an optimized resource configuration, wherein, Is the configuration of the resource after the optimization,Is a weight parameter that is used to determine the weight of the object,The proportion is adjusted to be a certain value,AndRepresenting the assessed resource parameters and the current resource configuration, respectively.
5. The intelligent sound box control system according to claim 4, wherein the step of obtaining the audio status adjustment result specifically comprises:
based on the optimized resource configuration, activating audio capture, collecting environmental audio data, and adopting the formula:
；
a captured audio data parameter is generated, wherein, Representing the captured audio data and,AndIs the gain and sensitivity adjustment coefficient,Is the ambient audio intensity;
inputting the captured audio data parameters into a Kalman filter, analyzing and updating the state of an audio signal, adjusting the smoothness and response speed of the signal, and adopting the formula:
；
a filtered audio signal is generated, wherein, Representing the audio signal after the filtering,And gamma is the sensitivity and threshold that regulates signal processing,Representing captured audio data;
based on the filtered audio signal, updating the audio state by adopting the formula:
；
an audio state adjustment result is generated, wherein, Is the result of the audio state adjustment,AndThe update rate and adjustment range of the audio state are adjusted,Representing the filtered audio signal.
6. The intelligent sound box control system according to claim 5, wherein the step of obtaining the optimized voice capturing result specifically comprises:
According to the audio state adjustment result, real-time noise suppression is executed, audio capturing in the environment is optimized, and the formula is adopted:
；
generating noise-suppressed audio parameters, wherein, In order to optimize the noise suppression parameters after the optimization,Noise suppression gain, stability coefficient and sensitivity adjustment parameters,Representing an audio state adjustment result;
And capturing and primarily processing the voice command controlled by the intelligent sound box by utilizing the audio parameters after noise suppression, wherein the voice command is represented by the formula:
；
generating the voice parameters after preliminary processing, wherein, For the initially processed voice command parameters,AndIs the processing gain and weight parameters that are used,In order to optimize the noise suppression parameters after the optimization,Is the original voice data in the environment;
Updating the voice recognition configuration of the intelligent sound box according to the primarily processed voice parameters, and adopting the formula:
；
generating an optimized speech capture result, wherein, In order to optimize the result of the speech capture,The update coefficient is represented by a number of coefficients,Is the voice instruction parameter after preliminary processing.
7. The intelligent sound box control system according to claim 6, wherein the step of obtaining the sound box performance log specifically comprises:
And optimizing the representation of response time by using the optimized voice capturing result through the formula:
；
a response time performance index is generated, wherein, AndRepresenting the gain factor, the base bias and the adjustment factor,Representing the performance index of the response time,Representing the optimized voice capturing result;
and combining the response time performance index and the processing load, adopting the formula:
；
Generating a performance log parameter, wherein, As a function of the performance log parameters,AndIs a weight parameter that is used to determine the weight of the object,Representing the performance index of the response time,Representing a processing load;
using the performance log parameters, by configuring the sound box parameters and recording key performance data, the formula is adopted:
；
generating a sound box performance log, wherein, Is the coefficient of fusion and is used for the fusion,Is a log of the previous performance of the device,Is the performance log of the sound box,Is a performance log parameter.
8. The intelligent sound box control system according to claim 7, wherein the step of obtaining the adjusted sound box performance regulation response configuration specifically comprises:
extracting key performance indexes from the sound box performance log, wherein the key performance indexes comprise response efficiency and processing load, calculating preliminary performance data, and adopting the formula:
；
generating a performance analysis result, wherein, Representing performance data extracted from the log, 、The weight coefficient of the weight of the sample,Representing the processing load of the device,Representing response efficiency;
Based on the performance analysis result, processing parameters of the sound box are adjusted, and the response speed of the equipment is optimized, through the formula:
；
an optimized processing configuration is generated, wherein, In order to optimize the configuration of the process,AndIs the adjustment parameter of the device, which is used for adjusting the parameters,Indicating that the processing parameters of the sound box are adjusted,Representing performance data extracted from the log;
Updating the performance regulation configuration of the sound box by using the optimized processing configuration, and applying the formula:
；
Obtaining the adjusted sound box performance regulation response configuration, wherein, Is the update coefficient of the coefficient,Is the prior sound box performance regulation configuration,In order to optimize the configuration of the process,And representing the adjusted sound box performance regulation response configuration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411096568.8A CN118645101A (en) | 2024-08-12 | 2024-08-12 | Intelligent sound box control system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411096568.8A CN118645101A (en) | 2024-08-12 | 2024-08-12 | Intelligent sound box control system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118645101A true CN118645101A (en) | 2024-09-13 |
Family
ID=92669827
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411096568.8A Pending CN118645101A (en) | 2024-08-12 | 2024-08-12 | Intelligent sound box control system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118645101A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008080912A1 (en) * | 2007-01-04 | 2008-07-10 | International Business Machines Corporation | Systems and methods for intelligent control of microphones for speech recognition applications |
| US10409551B1 (en) * | 2016-06-21 | 2019-09-10 | Amazon Technologies, Inc. | Voice-driven monitoring of resources in a service provider network |
| CN115762516A (en) * | 2022-11-09 | 2023-03-07 | 晨雨初听(武汉)文化艺术传播有限公司 | Man-machine interaction control method, equipment and storage medium |
| CN117854739A (en) * | 2024-03-05 | 2024-04-09 | 天津市第二人民医院（天津市传染病医院） | Intelligent internal medicine nursing monitoring system |
| CN117894311A (en) * | 2024-02-29 | 2024-04-16 | 大连博涛文化科技股份有限公司 | Bionic robot in aspect of voice recognition control |
| CN118211779A (en) * | 2024-03-12 | 2024-06-18 | 广东铧鸿机电工程有限公司 | Intelligent security comprehensive management platform |
-
2024
- 2024-08-12 CN CN202411096568.8A patent/CN118645101A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008080912A1 (en) * | 2007-01-04 | 2008-07-10 | International Business Machines Corporation | Systems and methods for intelligent control of microphones for speech recognition applications |
| US10409551B1 (en) * | 2016-06-21 | 2019-09-10 | Amazon Technologies, Inc. | Voice-driven monitoring of resources in a service provider network |
| CN115762516A (en) * | 2022-11-09 | 2023-03-07 | 晨雨初听(武汉)文化艺术传播有限公司 | Man-machine interaction control method, equipment and storage medium |
| CN117894311A (en) * | 2024-02-29 | 2024-04-16 | 大连博涛文化科技股份有限公司 | Bionic robot in aspect of voice recognition control |
| CN117854739A (en) * | 2024-03-05 | 2024-04-09 | 天津市第二人民医院（天津市传染病医院） | Intelligent internal medicine nursing monitoring system |
| CN118211779A (en) * | 2024-03-12 | 2024-06-18 | 广东铧鸿机电工程有限公司 | Intelligent security comprehensive management platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109087669B (en) | Audio similarity detection method and device, storage medium and computer equipment | |
| WO2021139327A1 (en) | Audio signal processing method, model training method, and related apparatus | |
| CN109801646B (en) | Voice endpoint detection method and device based on fusion features | |
| CN117294985B (en) | TWS Bluetooth headset control method | |
| CN111667838B (en) | Low-power-consumption analog domain feature vector extraction method for voiceprint recognition | |
| CN113823275A (en) | Voice recognition method and system for power grid dispatching | |
| CN111489763B (en) | GMM model-based speaker recognition self-adaption method in complex environment | |
| KR101065188B1 (en) | Apparatus and method for speaker adaptation by evolutional learning, and speech recognition system using thereof | |
| CN109616098A (en) | Voice endpoint detection method and device based on frequency domain energy | |
| CN113132193B (en) | Control method and device of intelligent device, electronic device and storage medium | |
| CN117762372A (en) | Multi-mode man-machine interaction system | |
| CN116013359A (en) | Power equipment abnormal condition monitoring method and device based on two-stage voiceprint recognition algorithm | |
| CN117059068A (en) | Speech processing method, device, storage medium and computer equipment | |
| CN118486297A (en) | Response method based on voice emotion recognition and intelligent voice assistant system | |
| CN118645101A (en) | Intelligent sound box control system | |
| CN117520828B (en) | Audio processing method, device, equipment and storage medium | |
| CN113012717A (en) | Emotional feedback information recommendation system and method based on voice recognition | |
| CN112382293A (en) | Intelligent voice interaction method and system for power Internet of things | |
| CN118016106A (en) | Elderly emotion health analysis and support system | |
| CN118173092A (en) | Online customer service platform based on AI voice interaction | |
| US20230136220A1 (en) | Quantifying Signal Purity by means of Machine Learning | |
| CN115132195B (en) | Voice wakeup method, device, equipment, storage medium and program product | |
| CN113763978B (en) | Voice signal processing method, device, electronic equipment and storage medium | |
| JP5147012B2 (en) | Target signal section estimation device, target signal section estimation method, target signal section estimation program, and recording medium | |
| CN118368561B (en) | Bluetooth headset noise reduction processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |