CN113012682B - False wake-up rate determination method, device, apparatus, storage medium, and program product - Google Patents
False wake-up rate determination method, device, apparatus, storage medium, and program product Download PDFInfo
- Publication number
- CN113012682B CN113012682B CN202110313693.XA CN202110313693A CN113012682B CN 113012682 B CN113012682 B CN 113012682B CN 202110313693 A CN202110313693 A CN 202110313693A CN 113012682 B CN113012682 B CN 113012682B
- Authority
- CN
- China
- Prior art keywords
- wake
- model
- target
- awakening
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012360 testing method Methods 0.000 claims abstract description 38
- 238000012549 training Methods 0.000 claims description 41
- 230000008569 process Effects 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 7
- 238000005457 optimization Methods 0.000 description 31
- 238000010586 diagram Methods 0.000 description 12
- 238000004590 computer program Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/01—Assessment or evaluation of speech recognition systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Traffic Control Systems (AREA)
Abstract
The disclosure discloses a false wake-up rate determining method, device, equipment, storage medium and program product, relating to the technical field of computers, in particular to the technical field of voice. The specific implementation scheme is as follows: determining the false wake-up frequency of the initial wake-up model in the candidate scene; selecting a target scene from the candidate scenes according to the false wake-up frequency of the candidate scenes; optimizing the target scene of the initial awakening model to obtain a target awakening model; and determining the false wake-up rate of the candidate scene based on the target wake-up model. According to the technical scheme of the embodiment of the disclosure, the samples for testing the false wake-up rate are enriched, and the accuracy of testing the false wake-up rate is improved.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, a storage medium, and a program product for determining a wake-up rate.
Background
With the rapid development of intelligent voice technology, more and more people begin to use intelligent voice applications, for example, voice assistants used during driving, and the intelligent voice technology brings great convenience to the lives of people.
In the process of using the intelligent voice application, the false awakening rate is one of important indexes influencing the use feeling of the user, and the user is troubled to a certain extent due to the high false awakening rate, so that the false awakening rate test is particularly important in the test process of the intelligent voice application.
Disclosure of Invention
The disclosure provides a method, a device, equipment and a storage medium for determining a false wake-up rate.
According to an aspect of the present disclosure, there is provided a false wake-up rate determining method, including:
determining the false wake-up frequency of the initial wake-up model in the candidate scene;
selecting a target scene from the candidate scenes according to the false wake-up frequency of the candidate scenes;
optimizing the target scene of the initial awakening model to obtain a target awakening model;
and determining the false wake-up rate of the candidate scene based on the target wake-up model.
According to another aspect of the present disclosure, there is provided a false wake-up rate determination apparatus, the apparatus including:
the false wake-up frequency determining module is used for determining the false wake-up frequency of the initial wake-up model in the candidate scene;
the target scene selection module is used for selecting a target scene from the candidate scenes according to the false wake-up frequency of the candidate scenes;
the target awakening model obtaining module is used for optimizing the target scene of the initial awakening model to obtain a target awakening model;
and the false wake-up rate determining module is used for determining the false wake-up rate in the candidate scene based on the target wake-up model.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method of any one of the embodiments of the disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of any one of the embodiments of the present disclosure.
According to the technology disclosed by the invention, the sample for testing the false awakening rate is enriched, and the accuracy of testing the false awakening rate is improved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is a schematic diagram of a false wake-up rate determination method according to an embodiment of the disclosure;
FIG. 2a is a schematic diagram of a false wake-up rate determination method according to an embodiment of the disclosure;
FIG. 2b is a flow chart of a false wake rate determination method according to an embodiment of the present disclosure;
FIG. 3a is a schematic diagram of a false wake rate determination method according to an embodiment of the disclosure;
FIG. 3b is a schematic diagram of a candidate scene partition according to an embodiment of the disclosure;
FIG. 4 is a schematic diagram of a false wake-up rate determination apparatus according to an embodiment of the present disclosure;
fig. 5 is a block diagram of an electronic device for implementing the false wake-up rate determination method of the embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of a false wake-up rate determining method in the embodiment of the present disclosure, where a technical solution of the embodiment of the present disclosure is suitable for a situation where wake-up model optimization and false wake-up rate determination are performed through scene division, and the method may be executed by a false wake-up rate determining device, where the device may be implemented in a software and/or hardware manner, and may be generally integrated in an electronic device, for example, a terminal device, and the method in the embodiment of the present disclosure specifically includes the following steps:
and S110, determining the false wake-up frequency of the initial wake-up model in the candidate scene.
The initial wake-up model refers to a current version wake-up model before optimization and update, and may refer to a wake-up model in an internal test phase in this embodiment. The wake-up model can be tested under the line, the internal test stage, the model optimization updating and the test and optimization of a plurality of stages of the on-line test.
The false wake-up is a case where the user inputs a voice not intended for wake-up while using the wake-up model, and the wake-up model is woken up. For example, when a user uses an electronic map application in the driving process, the user talks with surrounding people to mention that the user goes to a location A, the electronic map application is awakened and starts a navigation function to the location A after the audio is collected by the electronic map application in the starting state, and the condition is false awakening.
In the embodiment of the disclosure, in order to optimize the initial wake-up model in a targeted manner, multiple use scenes of the initial wake-up model may be used as candidate scenes, and the false wake-up frequency in each candidate scene may be obtained. Illustratively, firstly, a false wake-up audio generated in an internal test process of an initial wake-up model is obtained, candidate scenes corresponding to the false wake-up audio are searched in a user log, and then the false wake-up frequency in each candidate scene is determined. Specifically, for an initial wake-up model used in an electronic map application, the candidate scenes may include a scene in a navigation state and a scene in a non-navigation state; and performing candidate scene division with finer granularity, including scene division by pages, for example, the scene in the navigation state can be divided into a scene of a driving navigation page, a scene of a walking navigation page, a scene of a riding navigation page, and the like.
And S120, selecting a target scene from the candidate scenes according to the false wake-up frequency of the candidate scenes.
In the embodiment of the disclosure, after determining the false wake-up frequency of the initial wake-up model in a plurality of candidate scenes, a target wake-up scene may be selected from the candidate scenes according to the false wake-up frequency corresponding to each candidate scene, where the target wake-up scene is used to perform targeted periodic optimization on the wake-up model, specifically, a candidate scene with the highest false wake-up frequency in the candidate scenes may be selected as the target scene, and at least one candidate scene with the false wake-up frequency higher than a set threshold in the candidate scenes may also be selected as the target scene, which is not specifically limited herein.
Illustratively, in an electronic map application, a plurality of navigation Page scenes such as a driving navigation Page, a walking navigation Page, a riding navigation Page and the like which generate a false wake-up condition, and a plurality of non-navigation Page scenes such as a front Page, a route Page and a peripheral location Page are included, the scenes are all used as candidate scenes, a target wake-up scene can be selected according to a false wake-up PV (Page View) magnitude corresponding to the candidate scenes, specifically, the optimization priority of each candidate scene can be determined according to the PV magnitude, and finally, one or more candidate scenes with higher optimization priority are selected as the target scenes according to a set screening rule, so that targeted model optimization based on the target scenes is realized, the workload of acquiring training samples in the model optimization is reduced, and the efficiency of staged model optimization is improved.
S130, optimizing the target scene of the initial awakening model to obtain a target awakening model.
In the embodiment of the disclosure, after at least one target scene with a high false wake-up frequency is determined, in order to reduce the false wake-up rate of the wake-up model in the target scene, the initial wake-up model may be optimized for the target scene to obtain an optimized target wake-up model, specifically, in the wake-up model training process, the proportion of the training samples corresponding to the target scene to all the training samples may be increased, thereby implementing model optimization for the target scene, making the model optimization more targeted, and when performing model optimization each time, selecting part of candidate scenes as the target scene for performing model optimization, which may reduce the workload of the staged model optimization, increase the efficiency of the model optimization in each stage, and accelerate the product online speed.
Illustratively, in the electronic map application, when the target scene is acquired as a driving navigation page scene, it is indicated that the number of times of mistaken awakening is large when a user uses the driving navigation page, and the user experience is easily affected, and the model optimization for the target scene can be realized by increasing the proportion of model training samples in all training samples in the scene, so as to obtain the optimized target awakening model.
And S140, determining the false wake-up rate in the candidate scene based on the target wake-up model.
In the embodiment of the disclosure, after the optimized target wake-up model is obtained, the target wake-up model can be subjected to the false wake-up rate test again, so that the false wake-up rate of the target wake-up model in a candidate scene is determined, specifically, the false wake-up audio generated in the internal test process can be input into the SDK packaging the optimized target wake-up model for the false wake-up rate test, on one hand, the false wake-up audio generated in the internal test process can be used as a sample for testing the false wake-up rate, the comprehensiveness of the sample is improved, on the other hand, the false wake-up audio is directly input into the SDK, the interference caused by the external environment is avoided, and the accuracy of the obtained false wake-up rate is improved.
According to the technical scheme, the false wake-up frequency of the initial wake-up model in the candidate scene is determined, the target scene is selected from the candidate scene according to the false wake-up frequency of the candidate scene, then the target scene is optimized on the initial wake-up model to obtain the target wake-up model, finally the false wake-up rate in the candidate scene is determined based on the target wake-up model, samples for testing the false wake-up rate are enriched, and the accuracy of testing the false wake-up rate is improved.
Fig. 2a is a schematic diagram of a false wake-up rate determination method in the embodiment of the present disclosure, which is further refined on the basis of the above embodiment, and provides a specific step of determining a false wake-up frequency of an initial wake-up model in a candidate scene, a specific step of performing target scene optimization on the initial wake-up model to obtain a target wake-up model, and a specific step of determining a false wake-up rate in the candidate scene based on the target wake-up model. With reference to fig. 2a, a method for determining a false wake-up rate according to an embodiment of the present disclosure is described below, including the following steps:
s210, obtaining a first historical awakening audio and a first historical awakening result of the initial awakening model in a first use stage.
The first use phase may be understood as a use phase of the initial wake-up model in an internal test process, and exemplarily, the first use phase is a use process of the initial wake-up model performed by a test user for a test purpose.
In the embodiment of the disclosure, a first historical awakening audio generated by a user at a first use stage of an initial awakening model is acquired, and then a first historical awakening result aiming at the first historical awakening audio is acquired in a user log according to an audio identifier of the first historical awakening audio, wherein the user log is a log containing a use situation generated when the user uses the initial awakening model. The first historical awakening audio is the audio which is input by a test user in the process of using the initial awakening model and is used for awakening the initial awakening model, the first historical awakening result is a response result output by the initial awakening model after the first historical awakening audio is input into the initial awakening model, and the response result comprises two conditions of awakening and not awakening, and under the awakening condition, the response result also comprises two conditions of normal awakening and mistaken awakening.
S220, determining a candidate scene to which the first historical awakening audio belongs.
In the embodiment of the disclosure, after the first historical wake-up audio of the initial wake-up model in the first use stage is obtained, the candidate scene to which the first historical wake-up audio belongs is determined, and specifically, the candidate scene corresponding to the audio identifier may be searched in the user log according to the audio identifier corresponding to at least one first historical wake-up audio, where the candidate scene may be a scene divided in the functional dimension of the application or a scene divided in the page dimension of the application.
For example, in an electronic map application, the functional dimension may be divided into a scene of a navigation state and a scene of a non-navigation state, and the page dimension may further divide the scene of the non-navigation state into a scene of a top page, a scene of a route page, and other page scenes such as a non-navigation page. The candidate scene corresponding to the audio identifier may be searched in the user log according to the audio identifier of the first historical wake-up audio, for example, a scene in which the scene corresponding to the first historical wake-up audio is in a navigation state is obtained according to the audio identifier.
And S230, determining the false wake-up frequency of the initial wake-up model in the candidate scene according to the first historical wake-up audio, the first historical wake-up result and the candidate scene to which the first historical wake-up audio belongs.
In the embodiment of the disclosure, after the first historical wake-up audio, the first historical wake-up result and the candidate scene to which the first historical wake-up audio belongs are obtained, according to the first historical wake-up result, a part of the first historical wake-up audio, which belongs to the false wake-up audio, in the first historical wake-up audio is extracted, according to the candidate scene corresponding to the part of the first historical wake-up audio, the false wake-up frequency of the initial wake-up model in each candidate scene is determined, and by determining the false wake-up frequency of each candidate scene, the weak link of the initial wake-up model can be determined, that is, the false wake-up frequency of the initial wake-up model in which candidate scenes is higher is determined, so that a direction is provided for optimization of the initial wake-up model.
For example, in the application of the electronic map, a part of the first historical wake-up audio belonging to false wake-up is firstly identified in the first historical wake-up audio, and then the false wake-up frequency under each navigation page is determined according to the navigation page to which the screened first historical wake-up audio belongs.
And S240, selecting a target scene from the candidate scenes according to the false wake-up frequency of the candidate scenes.
S250, selecting a training sample of the target scene from a second historical awakening audio and a second historical awakening result of the initial awakening model in a second use stage; wherein the second phase of use is different from the first phase of use.
In the embodiment of the disclosure, after at least one target scene with a high frequency of false awakening is determined, in order to perform targeted optimization on the initial awakening model, a training sample of the target scene is selected from a second historical awakening audio and a second historical awakening result of the initial awakening model in a second use stage, specifically, the second historical awakening audio which is generated by false awakening is determined according to the second historical awakening result, a scene corresponding to the second historical awakening audio which is generated by false awakening is further obtained, and the false awakening audio of which the corresponding scene is the target scene is finally obtained as the training sample of the initial awakening model, where the second use stage may be a user use stage different from the first use stage, and exemplarily, the second use stage may be a user use stage before or after the first use stage.
Still taking the electronic map application as an example, as shown in fig. 2b, when it is determined that the target scene is the driving navigation page and the top page, in the second use stage, the second historical wake-up audio input by the user on the driving navigation page and the top page may be selected, and the second historical wake-up audio that is mistakenly waken up is taken as the training sample of the initial wake-up model.
And S260, training the initial awakening model by adopting the training sample to obtain a target awakening model.
In the embodiment of the disclosure, after training samples for a target scene are acquired in a second historical awakening audio in a second use stage, the training samples are input to an initial awakening model for model training, so as to obtain an optimized target awakening model. Specifically, the proportion of training samples corresponding to the target scene in all the training samples can be increased, targeted model training is achieved, and the false awakening rate of the initial awakening model in the target scene is reduced.
It is worth noting that after the initial awakening model is optimally trained by using the training sample aiming at the target scene, the training samples in other scenes except the target scene in the second use stage can be input into the optimized target awakening model, and whether the false awakening rate of the optimized target awakening model in other scenes is high or not is verified, so that the false awakening rate of the target awakening model in the target scene is reduced on the premise that the false awakening rate of other scenes is not increased.
And S270, taking the first historical awakening audio as the input of the target awakening model to obtain a target awakening result output by the target awakening model.
In the embodiment of the disclosure, after the target wake-up model is obtained through training, the first historical wake-up audio belonging to the false wake-up audio in the first use stage is stored in the false wake-up sample library, and the first historical wake-up audio belonging to the false wake-up audio in the false wake-up sample library is input to the target wake-up model, so as to obtain the target wake-up result output by the target wake-up model. Exemplarily, the false wake-up audio applied to the electronic map in each scene in the first use stage is input to the target wake-up model, and the wake-up result output by the target wake-up model for each false wake-up audio is obtained. And storing the first historical awakening audio belonging to the mistaken awakening audio in the first use stage into the mistaken awakening sample library, and improving the comprehensiveness of the test sample by the mistaken awakening rate of the mistaken awakening sample test target awakening model in the mistaken awakening sample library.
For example, as shown in fig. 2b, a first historical wake-up audio in a first usage phase is input into a voice recognition SDK encapsulating a target wake-up model, the voice recognition SDK generates an audio stream including at least one first historical wake-up audio, the voice recognition SDK performs voice endpoint detection on the audio stream, identifies at least one segment of audio included in the audio stream, and finally inputs the at least one segment of audio into the wake-up model to obtain a wake-up result output by the target wake-up model. At least one section of audio input to the target awakening model corresponds to the label information, and the label information is specifically whether the section of audio belongs to the mistaken awakening audio or not. By inputting the first historical awakening audio frequency into the voice recognition SDK packaged with the target awakening model, the interference of an external environment in the mistaken awakening test process is avoided, and the mistaken awakening rate test accuracy is improved.
And S280, calculating the false awakening rate of the target awakening model in the candidate scene according to the target awakening result.
In the embodiment of the disclosure, after the target wake-up result of the target wake-up model for each wake-up audio is obtained, the false wake-up rate of the target wake-up model in the candidate scene is calculated according to the wake-up result and the label information corresponding to the wake-up audio.
According to the technical scheme of the embodiment, a first historical awakening audio and a first historical awakening result of an initial awakening model in a first use stage are obtained, a candidate scene to which the first historical awakening audio belongs is determined, then a target scene is selected from the candidate scene according to the first historical awakening audio, the first historical awakening result and the candidate scene to which the first historical awakening audio belongs, a training sample of the target scene is selected from a second historical awakening audio and a second historical awakening result of the initial awakening model in a second use stage, the initial awakening model is trained by adopting the training sample to obtain a target awakening model, the first historical awakening audio is used as the input of the target awakening model to obtain the target awakening result output by the target awakening model, the mistaken awakening rate of the target awakening model in the candidate scene is calculated according to the target awakening result, optimization of the awakening model is achieved, the awakening audio in the first use stage is used as the test sample of the optimized target awakening model, and the test sample enrichment and the test accuracy are improved.
Fig. 3a is a schematic diagram of a false wake-up rate determining method in the embodiment of the present disclosure, which is further refined on the basis of the above embodiment and provides a specific step of selecting a target scene from candidate scenes according to the false wake-up frequency of the candidate scenes. With reference to fig. 3a, a method for determining a false wake-up rate according to an embodiment of the present disclosure is described below, including the following steps:
s310, determining the false wake-up frequency of the initial wake-up model in the candidate scene.
And S320, taking the candidate scene with the false awakening frequency larger than the false awakening threshold as a target scene.
In the embodiment of the disclosure, after the false wake-up frequency of the initial wake-up model in a plurality of candidate scenes is determined, the candidate scenes with the false wake-up frequency greater than a preset false wake-up threshold are used as target scenes, so that the subsequent optimization and updating of the initial wake-up model for the target scenes are realized.
In an electronic map application, in a use phase for testing, the number of times of false wake-up occurring under a driving navigation page is 100 times, the number of times of false wake-up occurring under a walking navigation page is 150 times, the number of times of false wake-up occurring under a route page is 120 times, and a preset false wake-up threshold value is 130 times. When the number of the targets is multiple, the priority of the target scene can be set according to the number of times of false awakening, the awakening model is optimized according to the priority of the number of times of false awakening, optimization training of the awakening model is more targeted, model optimization is performed only on the target scene, version change of an initial awakening model is accelerated, and user experience is improved.
Optionally, after the false wake-up frequency of the initial wake-up model in multiple candidate scenes is determined, the candidate scene with the highest false wake-up frequency may also be directly used as the target scene, and the optimization of the initial wake-up model for the target scene is further implemented. For example, the candidate scenes with higher false wake-up frequency have higher priority, and then the initial wake-up model is subjected to targeted multi-stage optimization updating according to the priority of the candidate scenes.
Optionally, the candidate scenes comprise a navigation scene of the electronic map and other scenes except for navigation;
the navigation scene comprises at least two navigation sub-scenes, and the navigation modes of different navigation sub-scenes are different; the other scenes comprise at least two page sub-scenes, the page types of the different page sub-scenes being different.
In this optional embodiment, the candidate scenes may be divided in a functional dimension, specifically as shown in fig. 3b, in the electronic map application, the candidate scenes may include a navigation scene and other scenes except the navigation scene of the electronic map in the functional dimension; the candidate scenes may also be page dimensions, and in the electronic map application, the navigation scene may be further divided into at least two navigation sub-scenes, where the navigation modes of different navigation sub-scenes are different, for example, the navigation sub-scenes include a plurality of navigation sub-scenes such as a driving navigation page, a motorcycle navigation page, a walking navigation page, and a truck navigation page, and similarly, other scenes except the navigation scene may be divided into at least two page sub-scenes, and the page types of different page sub-scenes are different, for example, a plurality of page sub-scenes including a home page, a route page, and a peripheral page.
The method comprises the steps of dividing candidate scenes according to different dimensions, obtaining false wake-up frequency of an initial wake-up model in different dimensions, and optimizing the initial wake-up model in a targeted manner, wherein for example, when the candidate scenes are divided in functional dimensions, the false wake-up frequency in a navigation scene is higher than that of other scenes except navigation, the initial wake-up model can be optimized for the navigation scene, when the candidate scenes are divided in page dimensions, the false wake-up frequency in a driving navigation page is the highest, the initial wake-up model can be optimized for the driving navigation page, model optimization is more targeted, and optimization efficiency of the initial wake-up model is improved.
S330, optimizing the target scene of the initial awakening model to obtain a target awakening model.
And S340, determining the false awakening rate in the candidate scene based on the target awakening model.
According to the technical scheme, the false wake-up frequency of the initial wake-up model in the candidate scene is determined, the candidate scene with the false wake-up frequency larger than the false wake-up threshold is used as the target scene, then the target scene is optimized on the initial wake-up model to obtain the target wake-up model, and finally the false wake-up rate in the candidate scene is determined based on the target wake-up model, so that the model optimization is more targeted, and the optimization efficiency of the initial wake-up model is improved.
Fig. 4 is a schematic structural diagram of a false wake-up rate determining apparatus in an embodiment of the present disclosure, where the false wake-up rate determining apparatus 400 includes: the device comprises a false wake-up frequency determining module 410, a target scene selecting module 420, a target wake-up model obtaining module 430 and a false wake-up rate determining module 440.
A false wake-up frequency determining module 410, configured to determine a false wake-up frequency of the initial wake-up model in the candidate scene;
a target scene selection module 420, configured to select a target scene from the candidate scenes according to the false wake-up frequency of the candidate scenes;
a target wake-up model obtaining module 430, configured to perform optimization on the target scene on the initial wake-up model to obtain a target wake-up model;
a false wake-up rate determining module 440, configured to determine a false wake-up rate in the candidate scene based on the target wake-up model.
According to the technical scheme, the false wake-up frequency of the initial wake-up model in the candidate scene is determined, the target scene is selected from the candidate scene according to the false wake-up frequency of the candidate scene, then the target scene is optimized on the initial wake-up model to obtain the target wake-up model, finally the false wake-up rate in the candidate scene is determined based on the target wake-up model, samples for testing the false wake-up rate are enriched, and the accuracy of testing the false wake-up rate is improved.
Optionally, the false wake-up frequency determining module 410 includes:
the first historical awakening audio acquisition unit is used for acquiring a first historical awakening audio and a first historical awakening result of the initial awakening model in a first use stage;
the candidate scene determining unit is used for determining a candidate scene to which the first historical awakening audio belongs;
and the false wake-up frequency determining unit is used for determining the false wake-up frequency of the initial wake-up model in the candidate scene according to the first historical wake-up audio, the first historical wake-up result and the candidate scene to which the first historical wake-up audio belongs.
Optionally, the candidate scenes comprise a navigation scene of the electronic map and other scenes except for navigation;
the navigation scene comprises at least two navigation sub-scenes, and the navigation modes of different navigation sub-scenes are different; the other scene comprises at least two page sub-scenes, the page types of the different page sub-scenes being different.
Optionally, the target wake-up model obtaining module 430 includes:
the training sample acquisition unit is used for selecting a training sample of a target scene from a second historical awakening audio and a second historical awakening result of the initial awakening model in a second use stage; wherein the second use phase is different from the first use phase;
and the target awakening model obtaining unit is used for training the initial awakening model by adopting the training sample to obtain the target awakening model.
Optionally, the target scene selection module 420 is specifically configured to:
and taking the candidate scene with the false awakening frequency larger than the false awakening threshold value as a target scene.
Optionally, the false wake-up rate determining module 440 includes:
the target awakening result acquisition unit is used for taking the first historical awakening audio as the input of the target awakening model to obtain a target awakening result output by the target awakening model;
and the false wake-up rate determining unit is used for calculating the false wake-up rate of the target wake-up model in the candidate scene according to the target wake-up result.
The device for determining the false wake-up rate provided by the embodiment of the disclosure can execute the method for determining the false wake-up rate provided by any embodiment of the disclosure, and has corresponding functional modules and beneficial effects of the execution method.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 5 illustrates a schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 5, the apparatus 500 comprises a computing unit 501 which may perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM) 502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
A number of components in the device 500 are connected to the I/O interface 505, including: an input unit 506 such as a keyboard, a mouse, or the like; an output unit 507 such as various types of displays, speakers, and the like; a storage unit 508, such as a magnetic disk, optical disk, or the like; and a communication unit 509 such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
The computing unit 501 may be a variety of general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 501 performs the various methods and processes described above, such as the false wake rate determination method. For example, in some embodiments, the false wake rate determination method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the false wake rate determination method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the false wake rate determination method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (14)
1. A false wake-up rate determination method, comprising:
determining the false wake-up frequency of the initial wake-up model in the candidate scene; wherein the candidate scene refers to a use scene of an initial wake-up model;
selecting a target scene from the candidate scenes according to the false wake-up frequency of the candidate scenes;
optimizing the target scene of the initial awakening model to obtain a target awakening model;
determining a false wake-up rate in the candidate scene based on the target wake-up model;
the determining of the false wake-up frequency of the initial wake-up model in the candidate scene includes:
acquiring false wake-up audio generated in the internal test process of the initial wake-up model, searching candidate scenes corresponding to the false wake-up audio in a user log, and determining the false wake-up frequency under each candidate scene;
the optimizing the target scene of the initial awakening model to obtain a target awakening model comprises: in the process of awakening model training, the proportion of training samples corresponding to the target scene in all the training samples is improved;
the determining the false wake-up rate in the candidate scene based on the target wake-up model includes: and inputting the mistaken awakening audio generated in the internal test process into the SDK packaged with the optimized target awakening model to perform the mistaken awakening rate test.
2. The method of claim 1, wherein the determining false wake-up frequency of the initial wake-up model in the candidate scene comprises:
acquiring a first historical awakening audio and a first historical awakening result of the initial awakening model in a first use stage;
determining a candidate scene to which the first historical wake-up audio belongs;
and determining the false wake-up frequency of the initial wake-up model in the candidate scene according to the first historical wake-up audio, the first historical wake-up result and the candidate scene to which the first historical wake-up audio belongs.
3. The method of claim 1, wherein the candidate scenes comprise a navigation scene and other scenes except navigation of the electronic map;
the navigation scene comprises at least two navigation sub-scenes, and the navigation modes of different navigation sub-scenes are different; the other scenes comprise at least two page sub-scenes, and the page types of the different page sub-scenes are different.
4. The method of claim 2, wherein the optimizing the initial wake-up model for a target scenario to obtain a target wake-up model comprises:
selecting a training sample of the target scene from a second historical wake-up audio and a second historical wake-up result of the initial wake-up model in a second use stage; wherein the second phase of use is different from the first phase of use;
and training the initial awakening model by adopting the training sample to obtain a target awakening model.
5. The method of claim 1, wherein the selecting a target scene from the candidate scenes according to the false wake-up frequency of the candidate scenes comprises:
and taking the candidate scene with the false awakening frequency larger than the false awakening threshold value as the target scene.
6. The method of claim 2, wherein the determining a false wake-up rate at the candidate scene based on the target wake-up model comprises:
taking the first historical awakening audio as the input of the target awakening model to obtain a target awakening result output by the target awakening model;
and calculating the false awakening rate of the target awakening model in the candidate scene according to the target awakening result.
7. A false wake-up rate determination apparatus, comprising:
the device comprises a false wake-up frequency determining module, a false wake-up frequency determining module and a processing module, wherein the false wake-up frequency determining module is used for determining the false wake-up frequency of an initial wake-up model in a candidate scene, and the candidate scene refers to a use scene of the initial wake-up model;
the target scene selection module is used for selecting a target scene from the candidate scenes according to the mistaken awakening frequency of the candidate scenes;
the target awakening model obtaining module is used for optimizing the target scene of the initial awakening model to obtain a target awakening model;
the false wake-up rate determining module is used for determining the false wake-up rate in the candidate scene based on the target wake-up model;
the determining of the false wake-up frequency of the initial wake-up model in the candidate scene includes:
acquiring false wake-up audio generated in the internal test process of the initial wake-up model, searching candidate scenes corresponding to the false wake-up audio in a user log, and determining the false wake-up frequency under each candidate scene;
the optimizing the target scene of the initial awakening model to obtain a target awakening model comprises: in the process of awakening model training, improving the proportion of training samples corresponding to the target scene in all training samples;
the determining the false wake-up rate in the candidate scene based on the target wake-up model includes: and inputting the false wake-up audio generated in the internal test process into the SDK packaged with the optimized target wake-up model to perform false wake-up rate test.
8. The apparatus of claim 7, wherein the false wake-up frequency determination module comprises:
the first historical awakening audio acquisition unit is used for acquiring a first historical awakening audio and a first historical awakening result of the initial awakening model in a first use stage;
a candidate scene determining unit, configured to determine a candidate scene to which the first historical wake-up audio belongs;
and the false wake-up frequency determining unit is used for determining the false wake-up frequency of the initial wake-up model in the candidate scene according to the first historical wake-up audio, the first historical wake-up result and the candidate scene to which the first historical wake-up audio belongs.
9. The apparatus of claim 7, wherein the candidate scenes comprise a navigation scene and other scenes than navigation of the electronic map;
the navigation scene comprises at least two navigation sub-scenes, and the navigation modes of different navigation sub-scenes are different; the other scenes comprise at least two page sub-scenes, and the page types of the different page sub-scenes are different.
10. The apparatus of claim 8, wherein the target wake model obtaining module comprises:
a training sample obtaining unit, configured to select a training sample of the target scene from a second historical wake-up audio and a second historical wake-up result of the initial wake-up model in a second usage phase; wherein the second phase of use is different from the first phase of use;
and the target awakening model obtaining unit is used for training the initial awakening model by adopting the training sample to obtain a target awakening model.
11. The apparatus of claim 7, wherein the target scenario selection module is specifically configured to:
and taking the candidate scene with the false awakening frequency larger than the false awakening threshold value as the target scene.
12. The apparatus of claim 8, wherein the false wake-up rate determination module comprises:
a target awakening result obtaining unit, configured to use the first historical awakening audio as an input of the target awakening model to obtain a target awakening result output by the target awakening model;
and the false wake-up rate determining unit is used for calculating the false wake-up rate of the target wake-up model in the candidate scene according to the target wake-up result.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110313693.XA CN113012682B (en) | 2021-03-24 | 2021-03-24 | False wake-up rate determination method, device, apparatus, storage medium, and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110313693.XA CN113012682B (en) | 2021-03-24 | 2021-03-24 | False wake-up rate determination method, device, apparatus, storage medium, and program product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113012682A CN113012682A (en) | 2021-06-22 |
| CN113012682B true CN113012682B (en) | 2022-10-14 |
Family
ID=76406096
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110313693.XA Active CN113012682B (en) | 2021-03-24 | 2021-03-24 | False wake-up rate determination method, device, apparatus, storage medium, and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113012682B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114120979A (en) * | 2022-01-25 | 2022-03-01 | 荣耀终端有限公司 | Optimization method, training method, device and medium of voice recognition model |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110310628A (en) * | 2019-06-27 | 2019-10-08 | 百度在线网络技术（北京）有限公司 | Wake up optimization method, device, equipment and the storage medium of model |
| CN111816178A (en) * | 2020-07-07 | 2020-10-23 | 云知声智能科技股份有限公司 | Voice equipment control method, device and equipment |
| CN112530418A (en) * | 2019-08-28 | 2021-03-19 | 北京声智科技有限公司 | Voice wake-up method, device and related equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210050003A1 (en) * | 2019-08-15 | 2021-02-18 | Sameer Syed Zaheer | Custom Wake Phrase Training |
-
2021
- 2021-03-24 CN CN202110313693.XA patent/CN113012682B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110310628A (en) * | 2019-06-27 | 2019-10-08 | 百度在线网络技术（北京）有限公司 | Wake up optimization method, device, equipment and the storage medium of model |
| CN112530418A (en) * | 2019-08-28 | 2021-03-19 | 北京声智科技有限公司 | Voice wake-up method, device and related equipment |
| CN111816178A (en) * | 2020-07-07 | 2020-10-23 | 云知声智能科技股份有限公司 | Voice equipment control method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113012682A (en) | 2021-06-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7269913B2 (en) | Knowledge graph construction method, device, electronic device, storage medium and computer program | |
| CN113342345A (en) | Operator fusion method and device of deep learning framework | |
| CN111488740A (en) | Causal relationship judging method and device, electronic equipment and storage medium | |
| CN113377890B (en) | Map quality inspection method and device, electronic equipment and storage medium | |
| CN112528641A (en) | Method and device for establishing information extraction model, electronic equipment and readable storage medium | |
| CN112560481B (en) | Statement processing method, device and storage medium | |
| CN113360711A (en) | Model training and executing method, device, equipment and medium for video understanding task | |
| CN113012682B (en) | False wake-up rate determination method, device, apparatus, storage medium, and program product | |
| CN112767935B (en) | Awakening index monitoring method and device and electronic equipment | |
| CN113157877B (en) | Multi-semantic recognition method, device, equipment and medium | |
| CN113808585A (en) | Earphone awakening method, device, equipment and storage medium | |
| CN112306452A (en) | Method, device and system for processing service data by merging and sorting algorithm | |
| CN115481594B (en) | Scoreboard implementation method, scoreboard, electronic equipment and storage medium | |
| CN113361621B (en) | Method and device for training model | |
| CN114254650A (en) | Information processing method, device, equipment and medium | |
| CN116244413B (en) | New intention determining method, apparatus and storage medium | |
| CN115131709B (en) | Video category prediction method, training method and device for video category prediction model | |
| CN115482809B (en) | Keyword retrieval method, keyword retrieval device, electronic equipment and storage medium | |
| CN113238765B (en) | Method, device, equipment and storage medium for distributing small program | |
| CN113360407B (en) | Function positioning method and device, electronic equipment and readable storage medium | |
| CN116737520B (en) | Data braiding method, device and equipment for log data and storage medium | |
| CN115860077B (en) | Method, device, equipment and storage medium for processing state data | |
| CN115758142A (en) | Deep learning model training method, data processing method and device | |
| CN117194018A (en) | Processing method and device of system temperature control algorithm in multi-core and multi-chip environment | |
| CN115482422A (en) | Deep learning model training method, image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |