CN111601215A - Scene-based key information reminding method, system and device - Google Patents

Abstract

The present invention provides a key information reminder method, system and embedded audio playback device. Compared with the prior art, the embedded audio playback device can independently complete scene-based continuous voice key information real-time detection, reminder, recording and playback. Convenience, convenience and privacy. The key information reminder system and reminder method are based on the actual needs of the current application scenario and the keywords or training samples customized by the user in advance to obtain a keyword recognition model that is highly suitable for the application scenario, so it can effectively improve the recognition of continuous speech streams. The accuracy of key information in the current application scenario, and timely output and save reminders for the information that should be focused on in the current application scenario and the information that users are interested in, the user experience is excellent.

Description

A scenario-based key information reminder method, system and device

technical field

The invention relates to the technical field of embedded devices, in particular to a scene-based key information reminder system, method and embedded audio playback device.

Background technique

At present, the Internet and mobile communication networks have entered thousands of households, covering every corner of people's lives. Various remote audio and video applications based on these remote communication platforms, such as online conferences, online teaching, online business negotiation, and online sales, are also accompanied by related technologies such as computer network technology, audio and video processing technology, and embedded devices with system-on-chip SoC as the core. And the high maturity of the product is rising day by day. These remote audio and video applications used with embedded devices such as mobile phones, earphones, tablet computers, speakers, etc., completely break the geographical restrictions, so that people in different places can realize real-time voice and video communication and interaction at any time, for people's production Life provides great convenience. For example, under the current turbulent epidemic situation, students can continue to teach at home through online teaching platforms. Students often participate in online classes through headsets and move around within a certain range while teaching. However, the disadvantage is that due to the lack of classroom atmosphere during online teaching, teachers cannot observe the status of each child's lectures in time, so they rely heavily on students' individual self-discipline. And once students lose their minds or play in private, no one can give timely reminders and corrections, and the course content taught by teachers is also missed. This kind of situation actually exists in the network video conference in a similar way. For example, the key voice information of the conference is missed due to interruption of personal affairs or answering the phone during the conference. Usually, there is no key information reminder function for the content of the opposite end speaker in the online courses and video conferencing software on the mobile phone or computer. Even if there are, local users are not necessarily on the side of mobile phones or computers. Therefore, it is very necessary to directly realize the function of key information reminder in the accessory devices such as headphones or speakers that are closest to the local user's mobile phone or computer, so that the local user's thinking can be pulled back to the network in the first time. class or video conference.

In recent years, speech recognition technology has been increasingly used in speech monitoring and identification of important information. Especially under the strong support of Moore's Law and big data, speech recognition based on artificial intelligence technology has entered the stage of deep learning from shallow recognition. Speech recognition technology based on deep learning theory and neural network model can output recognition results with higher accuracy, so it is widely used in many fields such as intelligent voice wake-up, intelligent voice control, and intelligent voice dialogue.

However, after in-depth research, the inventor found that if the artificial intelligence speech recognition technology is used in the current remote audio and video applications to realize the key voice information reminder function, there are many technical bottlenecks, such as:

First, in artificial intelligence speech recognition technology, the speech recognition model is the key to ensuring the recognition accuracy. However, in various applications of intelligent voice wake-up, intelligent voice control, and intelligent voice dialogue technology, the general-purpose voice recognition model is often used, that is, the training of the voice recognition model is completed in advance by the provider of the device/application, and the important information Judgment criteria and selection of training samples are all determined by the provider of the device/application. If this general-purpose speech recognition model is simply used in remote audio and video applications, it is difficult to adapt to various application scenarios, and may even lead to poor user experience because the recognition accuracy cannot be guaranteed.

Second, the implementation of artificial intelligence speech recognition technology, especially deep learning technology, requires a lot of high-precision computing, which depends on the strong support of hardware systems in terms of memory, computing overhead, and power consumption. Therefore, most of these technologies are currently used in GPU, FPGA and other high-cost, high-power, high-performance large-scale special-purpose computing platforms, and in various low-power, low-performance embedded devices that are most used by ordinary consumers. (such as headsets, portable speakers, telephone watches, conference terminal equipment, etc. mobile phone or computer accessory equipment) It is very rare to perform independent keyword recognition that does not depend on mobile phones or cloud, or only uses relatively simple isolated words or Fixed keyword sets, limited sentence pattern recognition and other technologies to achieve some simple and low-level speech recognition functions, such as simple voice wake-up, smart home voice control, etc., but failed to realize key voices in complex and continuous voice streams Information reminder function. For example, some of the well-known intelligent voice assistants on the market currently upload the voice stream collected by the embedded device to the mobile phone or the cloud for recognition. Usually, only a single sentence can be recognized. Uploading to the cloud or the remote device for identification and then returning the result usually results in poor user experience due to long delays, and it is difficult to protect the user's privacy. The reason is that one of the most important points is that the hardware computing power and power consumption of embedded devices are limited, and it is difficult to provide sufficient support for the current large-vocabulary continuous speech recognition technology.

On the third aspect, the current speech recognition in the consumer field is to perform a certain interaction after the local input speech stream is recognized by keywords or full speech. The function of reminding after voice recognition of interesting keywords.

Therefore, it is necessary to propose a scenario-based key information reminder technology to solve at least one of the above-mentioned technical defects.

SUMMARY OF THE INVENTION

In view of this, the present invention provides a key information reminder method, device, system and embedded audio playback device, which can effectively remind users to pay attention to key information.

In order to achieve the above object, as a first aspect of the present invention, an embedded audio playback device is provided, which includes a speaker and a communication unit, and also includes a control unit, a storage unit, a voice recognition unit and a reminder unit,

The communication unit receives the audio stream from the remote end;

The speech recognition unit includes a keyword recognition model unit, and the keyword recognition model unit is used to store a scene-based keyword recognition model;

The keyword is associated with the application scenario, and includes a set of vocabulary that needs to be focused on in the application scenario, and one or more of the vocabulary is pre-specified by the user;

The voice recognition unit extracts a voice signal from the audio stream, and uses the scene-based keyword recognition model to detect in real time whether the voice signal contains the keyword;

The control unit is configured to start recording the received audio stream when a keyword is included in the voice signal, and control the reminder unit to output key information reminders;

The storage unit is used to store the recorded audio stream;

The speaker is used to play the audio stream, or to play back the recorded audio stream in response to a playback instruction.

Preferably, the scene-based keyword recognition model may be pre-trained with a deep learning algorithm based on a training sample library containing speech samples for the keyword and/or speech samples of a specific person for the keyword. get;

The control unit may also be configured to download the scene-based keyword recognition model from a remote end through the communication unit.

Preferably, the speech recognition unit may further include a speech preprocessing unit, which is used for preprocessing the input audio stream to remove noise, background vocals, and musical sounds, and extract speech signals;

Preferably, the speech recognition unit may further include a neural network processing unit for performing data processing on the speech signal or the speech signal processed by the speech preprocessing unit by using a deep learning algorithm based on the keyword recognition model , so as to reason and judge the words appearing in the speech signal to determine whether the keyword words are contained in them.

Preferably, the reminder unit may be one or more of an indicator light module, a vibrator module, a text message generation module, a voice message generation module, and a music message generation module.

Further, an input unit may also be included for receiving a recording stop instruction and a playback instruction input by a user;

When the control unit contains keywords in the voice signal, it can start to continuously compress and encode the received audio stream and store it locally;

The control unit can stop the recording when receiving the recording stop instruction or when the continuous recording time exceeds the first predetermined duration;

The control unit can play the locally stored recording audio stream when receiving the playback local audio instruction;

Further, the control unit can also be used to send a recording start instruction to the far end when the voice signal includes a keyword, so that the far end starts to continuously record the audio stream sent, when the continuous recording time is not long. When the second predetermined duration is exceeded and a stop recording instruction is received, a recording stop instruction is sent to the remote end;

The control unit may send a playback request to the remote end, and receive and play the recorded audio stream stored at the remote end when receiving the instruction to play back the remote end audio.

Preferably, the embedded audio playback device is an earphone or a speaker with a call function.

As a second aspect of the present invention, a key information reminder system is provided, comprising an embedded audio playback device and a remote device,

The remote device receives a user-defined keyword vocabulary, and/or a user-provided voice sample of a specific person that contains at least the keyword, so as to obtain a scene-based keyword recognition model; the keyword Associated with the application scenario, which contains a set of vocabulary that needs to be focused on in the application scenario;

The scene-based keyword recognition model is obtained by training based on a training sample library containing speech samples for the keyword and/or speech samples of a specific person for the keyword in advance;

The embedded audio playback device communicates with the remote device, receives an audio stream from the remote device, and plays it;

The embedded audio playback device also obtains a voice signal from the audio stream, and uses a scene-based keyword recognition model to perform voice recognition on the voice signal, and detects in real time whether the voice signal contains keywords;

When the voice signal contains a keyword, the embedded audio playback device generates a reminder of key information and starts recording the received audio stream;

The embedded audio playback device plays the recorded audio stream in response to the playback instruction.

Further, a cloud server may also be included,

The remote device communicates with the cloud server, and sends the keyword and/or the voice sample of a specific person to the cloud server;

The cloud server uses the received keywords and/or the voice samples of the specific person to expand its standard sample library to form a training sample library, and uses deep learning algorithm training to obtain the training sample library based on the training sample library. Scenario-based keyword recognition model;

The remote device receives the scene-based keyword recognition model from the cloud server, and downloads the scene-based keyword recognition model to the embedded audio playback device.

Preferably, the remote device uses the keyword vocabulary input by the user and/or the user-provided speech sample of a specific person that contains at least the keyword to expand the standard sample library to form a training sample library, and based on the The training sample library is described, and the scene-based keyword recognition model is obtained by training with a deep learning algorithm;

The remote device downloads the scene-based keyword recognition model to the embedded audio playback device.

As a third aspect of the present invention, a key information reminder method is provided, wherein,

Receive a user-defined keyword vocabulary, and/or a user-provided voice sample of a specific person that contains at least the keyword; the keyword is associated with an application scenario, and includes a set of items that need to be focused on in the application scenario vocabulary;

The scene-based keyword recognition model is obtained by training based on a training sample library comprising speech samples for the keyword and/or speech samples of a specific person for the keyword;

When receiving and playing an audio stream, obtain a voice signal from the audio stream;

Use the scene-based keyword recognition model to perform speech recognition on the voice signal, and detect in real time whether the voice signal contains keywords;

When a keyword is included in the voice signal, a key information reminder is generated, and the received audio stream is started to be recorded;

In response to the playback instruction, the recorded audio stream is played.

Preferably, extensive speech samples are collected in advance to form a standard sample library;

Acquiring at least a voice sample containing the keyword according to the keyword;

Expanding the speech samples containing the keywords and/or the speech samples of the specific person to the standard sample library to form a training sample library, and using deep learning algorithm training based on the training sample library to obtain the scene-based keyword recognition model.

Further, in the step of acquiring the voice signal from the audio stream, a preprocessing step of eliminating noise, music, and background vocals may also be included;

Preferably, using the scene-based keyword recognition model to perform speech recognition on the speech signal or the preprocessed speech signal, and detecting in real time whether the speech signal contains keywords, may specifically include: constructing a speech signal based on the speech signal. The deep learning neural network of the keyword recognition model is used, and the speech signal is continuously input into the deep learning neural network for data processing, so as to infer and judge the words appearing in the speech signal, and determine whether the keyword words are included.

Preferably, the recording of the received audio stream may specifically include: when the voice signal contains a keyword, starting to continuously compress and encode the received audio stream and store it locally;

When receiving a recording stop instruction or the continuous recording time exceeds the first predetermined duration, stop the local recording;

The playing the recorded audio stream in response to the playback instruction specifically includes: in response to the playback local audio instruction, playing the locally stored recorded audio stream.

Preferably, the recording of the received audio stream may specifically include: when the voice signal contains a keyword, sending a recording start instruction to the remote end, and the remote end starts to continuously record the sent audio stream, and performs remote recording. end storage;

When the continuous recording time does not exceed the second predetermined duration and a stop recording instruction is received, the recording stop instruction is sent to the remote end, and the remote end stops recording;

The playing the recorded audio stream in response to the playback instruction specifically includes: in response to the playback remote audio instruction, sending a playback request to the remote end, and receiving and playing the recorded audio stream stored at the remote end.

Preferably, the key information reminder may be a combination of one or more of visual reminders, tactile reminders and auditory reminders;

The visual reminder includes light effect reminder and remote text message reminder;

The tactile reminder includes a vibration reminder;

The auditory reminder includes voice reminder, music reminder

The beneficial effects of the present invention are: compared with the prior art, the embedded audio playback device provided by the present invention can independently complete the real-time detection, reminder, recording and playback of key information of continuous voice based on the scene, which is convenient and convenient to use and has good privacy. . The key information reminder system and reminder method provided by the present invention are based on the actual needs of the current application scenario and the keywords or training samples customized by the user in advance to train and obtain a keyword recognition model that is highly compatible with the application scenario, so that the recognition model can be effectively improved. The accuracy of key information in the continuous voice stream, and timely output reminders for the information that should be focused on in the current application scenario and the information that users are interested in, and the user experience is excellent.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

Fig. 1 is the flow chart of the key information reminding method of Embodiment 1 of the present invention;

2 is a block diagram of the circuit principle of the embedded audio playback device according to Embodiment 2 of the present invention;

FIG. 3 is a system architecture diagram of a key information reminder system according to Embodiment 3 of the present invention.

Detailed ways

In the more than 40 years of Moore's Law application, semiconductor chip design technology and manufacturing process level have been rapidly improved, chip computing power has been greatly improved, on-chip storage capacity has been greatly improved, and power consumption has been continuously reduced, which makes artificial intelligence technology in small Widespread application in low-power embedded devices has become possible. The present invention is a technical improvement proposed in view of the defect in the prior art that people easily miss important information from the opposite end when using remote audio and video applications. Specifically, on the embedded device, for voice information, scene-based artificial intelligence speech recognition technology is used to identify interesting information from the opposite end in real time, and timely output reminders and save key audio streams. The present invention can be applied to different application scenarios and meet the individual needs of different users, so it can effectively solve the defects of the prior art. The "real-time" mentioned in this article means that the embedded audio playback device has enough computing power to recognize the keywords in the audio stream played at the original speed.

The technical solutions of the present invention will be further exemplarily described below through the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other in the case of no conflict.

Example 1:

As shown in FIG. 1, according to the core idea of the present invention, this embodiment provides a key information reminder method, wherein,

Step 100, an initialization step.

This step is the processing flow before entering the key information reminder, and is mainly used to check and update the software and hardware environment configuration, parameter preparation, program preparation, etc. required for the key information reminder. It may include the step of establishing a communication connection between the local device and the remote device by means of wireless communication or wired communication, and may also include the step of acquiring a scene-based keyword recognition model.

It should be noted that "local" and "remote" referred to in this article are relative concepts, where "local" refers to the party or end that receives the audio stream and generates key information reminders, and "remote" refers to an independent "Local" but communicate with "local" either directly or indirectly through one or more intermediaries, wired or wireless, and to the other party or end of the audio stream to "local". In addition, it should be noted that the term "remote end" and "peer end" commonly used in describing voice calls in this field are not the same concept. "Remote" refers to the initial receiver of the call audio stream from the peer end, and the corresponding "local" refers to the final receiver of the call audio stream.

As a specific implementation manner, "local" may be an audio playback device based on an embedded system (abbreviation: embedded audio playback device). The "embedded system" referred to in this article is a special-purpose computer system embedded in the object system. It is application-centric, based on computer technology, and can be tailored for software and hardware. , volume, power consumption has strict requirements for special computer systems. "Embedded device" refers to a device that contains an embedded system, which is generally based on ARM cores and architectures or other low-power cores and architectures, and is used to implement specific functions and applications. A device in terms of a PC, specifically a headset, a speaker, a phone watch, a conference terminal, and the like. And "remote" can be an end user computer system, a web server or server system, a mobile computing device, a consumer electronic device or other suitable electronic device or any combination or part thereof, such as a cell phone, tablet, computer, smart TV Wait.

Remote audio and video applications can be applied to various scenarios, and the amount of voice information transmitted is huge. The types of key information also vary from person to person, and from scenario to scenario. For example, people often participate in video conferences or online classes through headsets or speakers that are attached to mobile phones or computers. In a video conference, users may be most concerned about the part of the conference that involves themselves, such as their own department, their boss, their related business, etc., so they are used to identify key information. The words should be the name of the department, the name of the boss, the name of oneself, the name of the business, the assignment of tasks, the deadline for delivery, etc. In the online class, students may be most concerned about the knowledge points taught by the teacher, so they are used to identify key points. The keywords of the information should be the key points, difficulties, test points, summary, review, etc.; while in the customer service center, for the customer service staff, they may be most concerned about the complaints mentioned by customers, so the keywords need to include complaints, suggestions, quality , service attitude, etc. If a full-text speech recognition model is used in these different scenarios and the recognition accuracy is to be guaranteed, the speech recognition model must be trained based on a large number of speech samples. However, it is usually difficult to achieve this in accessory devices of mobile phones or computers. On the one hand, it is difficult to obtain massive voice samples. On the other hand, training based on massive voice samples has very high requirements on computer hardware and is expensive to implement. Costs limit the deployment of the technology in cellphone or computer accessory devices.

Therefore, in this embodiment, the step of acquiring the keyword recognition model based on the scene, especially the acquisition of the keyword recognition model in the accessory embedded device of the mobile phone or the computer, is used for adjusting according to the actual application scene. , Update the keyword recognition model to make the keyword recognition model more suitable for the current scene and meet user needs. The keywords are associated with application scenarios, and include a group of words that need to be focused on in the application scenarios. Different application scenarios can correspond to different keywords. Users can set and specify one or more words in the keywords according to actual needs.

The step of acquiring the scene-based keyword recognition model may specifically include: receiving a user-defined keyword vocabulary, and/or receiving a user-provided voice sample of a specific person that contains at least the keyword; and the voice samples of the specific person are used to expand the standard sample library, so as to form a training sample library, and a scene-based keyword recognition model is obtained by training based on the training sample library. Wherein, the standard sample library may be a training sample set formed based on pre-collected extensive speech samples.

The step of receiving a user-defined keyword vocabulary and/or receiving a user-provided voice sample of a specific person at least containing the keyword is usually performed remotely based on a richer user interface on a mobile phone or computer.

As an optional implementation, the user can set a custom keyword set through the remote end in advance according to his own preferences, needs, and usage scenarios; the provider of the audio stream can also use the scenario, audio stream content, user usage habits, etc. Various factors generate a default keyword set. The remote end can also display several default keyword words in advance for the user to select, add, and delete, so as to form a keyword word set associated with the application scenario.

In order to match the hardware environment of the embedded device, an upper limit of the number of words in the keyword can be set, for example, 30 groups of words and the like.

In addition, in view of speech recognition, various factors such as the speaker's gender, age, physiological characteristics of pronunciation, dialect, non-native pronunciation, emotion when speaking, environmental noise and other factors may affect the accuracy of recognition, for example, the same word " The key point”, the pronunciation of Sichuan and Cantonese are quite different. Therefore, in this embodiment, a voice sample with a specific accent provided by the user at least containing keywords can also be obtained, and the standard sample library can be expanded with it. For example, a student can provide some recordings of a teacher in class, and a staff member can provide a section of the boss’s meeting. recordings, etc.

After acquiring the keywords set by the user, the keywords are used to select the existing massive voice standard sample library to form a training sample library; After sampling, the training sample library is also augmented with the speech samples of the specific person. In this embodiment, a scene-based keyword recognition model is obtained by training a sample library containing these speech samples closely related to the application scene, which can effectively improve the recognition accuracy.

In the process of obtaining a keyword recognition model by training in this embodiment, the Hidden Markov Model (HMM), Dynamic Topic Models (DTM) and Various classical artificial intelligence speech recognition algorithms derived from such technologies can also be implemented by deep learning-based algorithms and various future related algorithms. Deep learning is one of the important fields in machine learning research. Its motivation is to build and simulate the neural network of the human brain for analysis and learning, and to interpret data, such as images, sounds and texts, by imitating the mechanism of the human brain. The core of deep learning is to learn more useful features by building a machine learning model with multiple hidden layers and a large amount of training data, thereby ultimately improving the accuracy of classification or prediction. At present, in computer vision and natural language, the mainstream deep learning algorithms are Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) algorithms, and there are also Long Short-term Memory Networks (Long Short-term Memory Networks). Term Memory, LSTM for short) algorithm, Deep Fully Convolutional Neural Network (DFCNN for short) algorithm, etc. As far as implementation is concerned, this embodiment may adopt various applicable deep learning algorithms including but not limited to these existing or future ones.

As a preferred implementation manner, this embodiment adopts a continuous speech keyword recognition technology based on a deep learning algorithm. For example, after obtaining the training sample library, deep learning algorithms such as a Convolutional Neural Network (CNN) algorithm and a Recurrent Neural Network (RNN) algorithm are used to train and obtain the training sample library based on the training sample library. Scenario-based keyword recognition model.

The training process of the keyword recognition model using the deep learning algorithm may be completed remotely, or may be completed in the cloud. It should be noted that the "cloud" mentioned in this article refers to the server side of cloud computing or the backend server of cloud computing with powerful processing and storage capabilities. As a preferred embodiment, the training process is completed in the cloud, so as to make full use of the hardware resources and powerful computing power of the cloud. Specifically, it includes: after the user inputs a keyword vocabulary at the remote end or uploads a voice sample of a specific person containing the keyword, the remote end sends the keyword and/or the voice sample to the cloud, so that the cloud can use various methods such as from the Internet Acquiring voice samples including the keywords, and expanding the voice samples and the voice samples of a specific person into its standard sample library to form a training sample library, and then using the training sample library to train to obtain scene-based keyword recognition Model.

Further, after the training is completed, the remote end receives the scene-based keyword recognition model from the cloud.

For the process of completing the training process at the remote end, reference may be made to the above-mentioned cloud training process, which will not be repeated here.

The initialization step may further include a step of updating the local keyword recognition model, which specifically includes: remotely downloading the scene-based keyword recognition model to the local. It may be that the remote end actively sends an update request to the local, or it may be that the remote end initiates the download execution in response to the local update request.

After completing the initialization steps, you can enter the following key information real-time detection and reminder process.

Step 110, when receiving and playing the audio stream, obtain the voice signal from the audio stream;

The key information reminding process of this embodiment is that in the remote audio and video application, while receiving and playing the audio stream in the remote audio and video application, the key information contained in the voice information is recognized and reminded.

As a preferred embodiment, when acquiring the voice signal in the audio stream, this step also performs background sound cancellation on the audio stream, removes noise, background human voice, music, etc. in the background, and extracts high signal noise. Compared with the foreground speech signal, to improve the signal-to-noise ratio, thereby improving the success rate of speech recognition.

Step 120, using the scene-based keyword recognition model to perform speech recognition on the voice signal, and detect in real time whether the voice signal contains a preset keyword;

When detecting whether the speech signal contains keywords, as long as a group of keyword words is detected, it can be considered that the speech signal contains keywords.

As a preferred embodiment, using the continuous speech keyword recognition technology based on the deep learning algorithm to recognize the key information based on the scene, specifically includes: constructing a deep learning neural network based on the keyword recognition model, The speech signal is input into the deep learning neural network for data processing, so as to infer and judge the words appearing in the speech signal, and determine whether the word contains a keyword or not.

In this embodiment, the continuous speech keyword recognition technology based on the deep learning algorithm is used to recognize the key information based on the scene. Compared with the continuous speech recognition of large vocabulary in the prior art, it is not necessary to recognize all the characters, but only detects a set of characters set by the user. Whether a group or groups of keywords appear in a continuous voice stream, on the one hand, it can be detected in real time for the continuous voice stream, on the other hand, it has low requirements on the computing power, storage space and power consumption of the hardware, and can be applied to small In low-power embedded systems, at the same time, scene-based recognition can effectively improve the recognition accuracy and improve the user experience of speech recognition.

When it is detected that the voice signal does not contain a keyword, the process returns to step 110 to continuously detect the subsequently acquired audio stream.

When the voice signal contains at least one group of keywords, perform steps 130 and 140;

Step 130, generating a key information reminder.

The key information reminders may include visual reminders, tactile reminders and auditory reminders;

The visual reminders include light effect reminders and text message reminders, such as LED indicators flashing or displaying specific light effects, flashing patterns appearing on the remote screen, and remote text messages (eg, notification messages of mobile phone applications APP), etc.;

The tactile reminder includes a vibration reminder, such as ringing according to a predetermined regularity;

The auditory reminders include voice reminders and music reminders, such as using predetermined voice content or music for reminders.

During specific implementation, one or more of the above-mentioned reminder methods can be selected according to the actual application scenario. For example, only light effect reminders or music reminders can be set, or a message can be sent to the associated computer application APP while the vibration reminder is performed. , in order to get the effect of double reminder.

Step 140, start recording the received audio stream;

In this embodiment, when judging that the voice information of the current audio stream contains keywords, in order to help the user to omit important content as little as possible, recording is also started on the received audio stream when a reminder is generated.

When recording the audio stream, the keyword itself can be used as the recording starting point, or the audio stream received after the keyword appears as the recording starting point, or a segment of the current audio stream when the keyword appears that has been scrolled and compressed and encoded. The audio stream starts from the keyword and moves forward by a fixed time as the recording start point. That is, the recorded audio stream may include the audio stream when the keyword appears, or may not include the audio stream before the keyword appears.

The recorded audio stream will be compressed and encoded and stored locally for local playback. The audio stream is continuously recorded until a recording stop instruction is received or when the continuous recording time exceeds the first predetermined duration, the recording is stopped. The first predetermined duration takes into account the limited capacity of the local storage carrier, and can be set to be relatively short, for example, 1 to 2 minutes. Usually, the important content will appear in the voice information within a short period of time after the keyword appears, so the shorter first predetermined duration, although short, may save the most important voice content, which is convenient for users to quickly understand when replaying the recording. Important information.

As a preferred way, when the voice signal contains a keyword, a recording start instruction can also be sent to the remote end, and the instruction can make the remote end start to continuously record the audio stream sent by it, and perform remote storage; After the recording start instruction is sent, the timer can also be started. When the calculated recording time of the far end does not exceed the second predetermined duration and the recording stop instruction is received, the recording stop instruction is sent to the far end, so that the far end can 2. The recording will be terminated at any time due to the receipt of the recording stop instruction within the predetermined time period, which increases the controllability of the recording time. The far end can automatically stop recording when the continuous recording time exceeds the second predetermined duration.

In order to help users fully grasp important information as much as possible and reduce information omission, the second predetermined duration can be set to be greater than or equal to the first predetermined duration, that is, to make the second predetermined duration longer, such as 2 to 5 minutes, so that the Longer audio streams of key information are saved for playback by the user.

Of course, if the local storage space is large enough, the first predetermined duration can also be set to be greater than or equal to the second predetermined duration. In this way, recordings with a sufficiently long duration are stored locally, while recordings with a shorter duration are retained at the remote end, so that the user can Or others can play back the recording at the far end to quickly understand key information.

In addition, as an optional implementation manner, the remote end may also perform full-text speech recognition on the recorded audio stream when recording the audio stream, so as to obtain the corresponding text, and store the text information.

Step 150, in response to the playback instruction, play the recorded audio stream.

In this step, the locally recorded and stored audio stream may be played in response to the playback local audio command; the playback request may also be sent to the remote end in response to the playback remote audio command, and the recorded audio stream stored at the remote end may be received and played.

As an optional implementation manner, when the recorded audio streams are stored locally, they may be stored in the order of the recording start time; correspondingly, during playback, they may be played in the order of the recording start time.

It should be noted that step 104 is a step performed based on the obtained playback specification, so it does not have to be implemented after step 103, or it can be implemented to detect playback instructions at any time during use to playback the recording.

In a typical application scenario, the key information reminding method of this embodiment can be applied to a call center system. Usually the operator of the call center has to answer hundreds of voice calls every day, which is very intensive. However, due to differences in oral expression ability, accent problems, and even emotional problems, it is often difficult for the caller to clearly express the main purpose of the call in a short period of time. If the operator cannot concentrate highly, it is easy to miss the important information of the other party, or even misunderstand the meaning of the other party, which will lead to bad consequences. Using the method of this embodiment, the operator wears a headset that can remind key information when answering a call, and the headset automatically identifies whether the voice information of the caller contains key information such as "alarm", "complaint", and "liar". In addition, the headset can also record key information, or notify the remote end (such as call center management platform, call transfer platform, etc.) to record key information. In this way, the operator can get a more accurate and comprehensive understanding of key information through the playback function, and deepen the understanding of the other party's call intention. It can be seen that the key information reminding method of this embodiment can not only remind the operator in a timely and effective manner, but also help the operator to review the content of the call, which not only reduces the loss of information, but also greatly reduces the operator's work pressure.

Example 2

2, according to the core idea of the present invention, the present embodiment provides an embedded audio playback device, including a communication unit, a speaker, a control unit, a storage unit, a voice recognition unit and a reminder unit,

The storage unit is used for storing data, programs and the like related to the operation of the device.

The communication unit may be a wired communication unit or a wireless communication unit, and may include both a wired communication module and a wireless communication module. Specifically, the communication unit can be implemented as a Bluetooth communication unit, a WIFi communication unit, an Internet network interface, an audio dedicated wired transmission interface, a USB interface, a micro USB interface, a mini usb interface, a Type-C interface, a Lightning interface, etc. Known or future available communication units in this embodiment.

The communication unit receives the audio stream from the remote end;

Described speech recognition unit is used for extracting speech signal from described audio stream, and adopt scene-based keyword recognition model to detect in real time whether described speech signal contains keyword;

The control unit is the control center of the device, which uses various interfaces and lines to connect other units in the device, and conducts overall monitoring and scheduling for each unit to realize various functions of the device, especially in the When the voice signal contains keywords, start recording the received audio stream, and control the reminder unit to output key information reminders;

In this embodiment, the keyword is associated with an application scenario, and includes a set of vocabulary that needs to be focused on in the application scenario, and one or more of the vocabulary is pre-specified by the user;

The speech recognition unit includes a keyword recognition model unit for storing the scene-based keyword recognition model. The scene-based keyword recognition model is obtained by training based on a training sample library containing speech samples for the keyword and/or speech samples of a specific person for the keyword in advance; as a preferred embodiment , the scene-based keyword recognition model is obtained by training with a deep learning algorithm, and the speech recognition unit can use the keyword recognition model to perform continuous speech keyword recognition to detect in real time whether the speech signal contains key words word;

The speech recognition unit may also include a speech preprocessing unit for preprocessing the input audio stream to eliminate noise, music, background vocals, etc., and extract a speech signal with a high signal-to-noise ratio;

The speech recognition unit may further include a neural network processing unit for performing data processing on the speech signal by using a deep learning algorithm based on the keyword recognition model, so as to infer and judge the words appearing in the speech signal. , to determine if it contains a keyword word. The neural network processing unit can be an embedded neural network processor (neural-network processunits, NPU), a dedicated neural network processing array processing unit, a DSP, an embedded processor, etc., which can be used to process massive multimedia data in a neural network. processing module.

In this embodiment, the keyword recognition model is externally trained and downloaded into the device before use. Therefore, the control unit is further configured to download the scene-based keyword recognition model from a remote end through the communication unit.

The reminding unit is one or more of an indicator light module, a vibrator module, a text message generation module, a voice message generation module, and a music message generation module. Wherein, the indicator light module can be an LED indicator light, which can output reminders in the form of flashing lights or displaying specific graphics; the vibrator module can generate vibrations with a predetermined frequency; the text message generating module can In order to generate a text message according to a predetermined message format, such as a text message containing currently recognized keywords; the voice message generation module can generate a voice message according to a predetermined voice message format, such as generating a currently recognized key word. The voice message of the word; the voice message generation module can select a segment from the pre-stored voice data according to a preset method, as a voice message, such as "DiDi", "Ding Dong" and other Tone sounds.

The speaker is used for playing the audio stream, or playing back the recorded audio stream, or playing the voice message or sound message, and so on. It should be understood that, in some specific implementation manners, the speaker may cooperate with the control unit and the storage unit to replace the function of the reminder unit, for example, only in the form of sound reminder.

The embedded audio playback device further includes an input unit for receiving various control instructions input by the user, for example, receiving a playback instruction, a stop reminder instruction, a recording stop instruction and the like input by the user.

The input unit may be various mechanical or voice input modules such as touch panels, buttons, and voice command input modules.

The storage unit is used to store the recorded audio stream;

In an optional implementation manner, when the voice signal contains a keyword, the control unit starts to continuously compress and encode the received audio stream and store it locally; when the control unit receives a recording stop instruction or When the continuous recording time exceeds the first predetermined time, the recording will be stopped;

The control unit plays the locally stored recording audio stream when receiving the playback local audio instruction;

The control unit is also used to send a recording start instruction to the far end when the voice signal contains a keyword, so that the far end starts to continuously record the sent audio stream, when the continuous recording time does not exceed the second predetermined time. When the duration is long and a stop recording instruction is received, a recording stop instruction is sent to the remote end;

The control unit sends a playback request to the remote end, and receives and plays the recorded audio stream stored at the remote end when receiving the remote end audio playback instruction.

In addition, the embedded audio playback device may further include a power supply unit, which is used to provide the required power supply when the device works, which may be a power supply circuit module powered by a button battery or a rechargeable battery, or a power supply circuit module powered by a button battery or a rechargeable battery. The power supply management module that supplies power to the device through an external input power supply can also be a circuit module that draws power by itself based on a wired communication interface.

Obviously, the embedded audio playback device of this embodiment can be used to implement some or all of the methods, processes or steps in the key information reminder method described in Embodiment 1. The descriptions of the same or similar parts as those in Embodiment 1 will not be repeated in this embodiment.

The embedded audio playback device can be implemented as a head-mounted audio playback device, such as various wired earphone devices, wireless earphone devices, etc., and can also be implemented as various portable speakers; it can also be implemented as a phone watch, Portable game consoles, portable multimedia players and other accessories for mobile phones or computers. For example, in a typical application scenario, the embedded audio playback device is a speaker with a call function. The casing of the speaker is provided with an LED indicator, and a scene-based keyword recognition model is pre-downloaded in the speaker, which can continuously detect the voice information currently played by the speaker in real time. When a keyword is included in the current voice message, the LED indicator starts flashing to alert the user. The speaker has an intelligent voice control function, and the user can issue control commands by voice to control the speaker to perform functions such as turning off the LED indicator, stopping recording, and playing back. For the detailed process of implementing the key information reminder by the speaker, reference may be made to the foregoing Embodiment 1 and part of the description in this embodiment, which will not be repeated here.

Example 3

According to the core idea of the present invention, this embodiment provides a key information reminder system, including an embedded audio playback device and a remote device,

The remote device receives a user-defined keyword vocabulary, and/or a user-provided voice sample of a specific person that contains at least the keyword, so as to obtain a scene-based keyword recognition model; the keyword Associated with the application scenario, which contains a set of vocabulary that needs to be focused on in the application scenario;

The scene-based keyword recognition model is obtained by training based on a training sample library containing speech samples for the keyword and/or speech samples of a specific person for the keyword in advance;

The embedded audio playback device communicates with the remote device, receives the audio stream from the remote device, and plays it; the communication can be in any suitable communication form, such as wired (eg ethernet, USB, lightning, optical fiber) ) communication or wireless (eg WiFi, Bluetooth, IR) communication.

The embedded audio playback device also obtains a voice signal from the audio stream, and uses a scene-based keyword recognition model to perform voice recognition on the voice signal, and detects in real time whether the voice signal contains keywords;

When the voice signal contains a keyword, the embedded audio playback device generates a reminder of key information and starts recording the received audio stream;

The embedded audio playback device plays the recorded audio stream in response to the playback instruction.

As an optional manner, the key recognition model is trained on the remote device, and the remote device uses a user-defined keyword vocabulary and/or a user-provided specific person who contains at least the keyword The voice sample is used to expand its standard sample library, form a training sample library, and obtain a scene-based keyword recognition model based on the training sample library training;

The remote device downloads the scene-based keyword recognition model to the embedded audio playback device.

As another optional implementation manner, the keyword recognition model is trained in the cloud, and the system further includes a cloud server;

The remote device communicates with the cloud server, and sends the keyword and/or the voice sample of a specific person to the cloud server;

The cloud server uses the received keywords and the voice samples of the specific person to expand its standard sample library, and obtains a scene-based keyword recognition model based on the training sample library;

The remote device receives the scene-based keyword recognition model from the cloud server, and downloads the scene-based keyword recognition model to the embedded audio playback device.

Obviously, the key information reminder system provided in this embodiment can be used to implement some or all of the methods, processes or steps in the key information reminder method described in Embodiment 1. The embedded audio playback device described in Embodiment 2 can also be used to implement the key information reminder system of this embodiment. For similar technical details, reference may be made to the descriptions of the foregoing embodiments, which will not be repeated here.

The following will take a typical application scenario as an example to describe the core idea of the embodiments of the present invention more clearly and in detail.

Referring to FIG. 3 , in this application scenario, the key information reminder system includes a video playback device (such as a tablet computer) 300 , an earphone 310 and a cloud server 320 .

The earphone 310 can be a headphone, an in-ear earphone or an ear-hook earphone; it can be a wired earphone or a wireless earphone; there can be only one earphone 311, and left and right earphones 311; the left and right earphones 311 can be It can be one-piece or split.

The headset 310 communicates with the video playback device 300 by wire or wirelessly, so as to receive the audio stream from the video playback device 300 . The video playback device 300 may be a user's personal computer, a tablet computer, a smart TV, a mobile phone, or the like. The user watches the video program through the video playback device 300 . Figure 3 shows students watching online classes through tablet computers.

The video playback device 300 may also access the cloud server 320 based on a network, and the network may be a local area network, a wide area network, a cellular network, or a combination thereof.

The earphone 310 is provided with an LED indicator 312 and buttons 313-316. The LED indicator 312 can emit a flashing red light; the button 313 is a volume up button, the button 314 is a play/pause button, the button 315 is a stop reminder/stop recording/playback button, and the button 316 is a volume down button. The button 315 can be set to perform three functions of stop reminder, stop recording and start playback at the same time by pressing it once, or it can be set to perform stop reminder and recording at the same time by pressing it once, and start playback by pressing it twice in a row. Specifically, it can be set according to the actual implementation environment, which is not specifically limited in the present invention.

The LED indicator 312 can also be set on the external microphone (not shown in the figure) of the headset 310. When wearing the headset, the user can adjust the external microphone to the position in front of his lips, so that the LED indicates The light 312 is like a light-emitting reminder, which is easier for the user to see.

In addition, a vibrator (not shown in the figure) is also provided in the earphone 310 . The vibrator can be implemented using existing or future applicable technologies, which is not specifically limited in the present invention. For example, it could be an eccentric motor with a cam.

The cloud server 320 may generate a keyword recognition model based on the aforementioned deep learning algorithm training. In a specific implementation, the cloud server 320 may collect a wide range of speech samples in advance, and perform vocabulary tagging and other processing on the speech samples to form a standard sample library.

In this application scenario, the key information reminder system realizes the process of key information reminder as follows:

Step 1, initialization step.

Before starting the key information reminder process, the initialization steps are carried out to check and update the software and hardware environment configuration and various parameter settings required for each device, equipment operation and communication in the system.

These include setting keywords and obtaining new keyword recognition models. Specifically:

The user sets the keyword vocabulary through the video playback device 300. For example, students can input words such as "focus", "exam", "summary" and their name as keywords before the online class. Through the user's independent settings, personalized keywords that conform to the current application scenario can be formed.

In order to match the hardware power consumption and computing power of the headset 310, the upper limit of the number of keyword words is set to 20.

When a new word is input into the keyword of the video playback device 300 , the video playback device 300 accesses the cloud server 320 , sends a request for updating the keyword recognition model to the cloud server 320 , and sends the keyword to the cloud server 320 .

After the cloud server 320 receives the keywords, it can compare the keyword words with the existing keyword words on the cloud server 320. When all words in the keywords sent by the video playback device 300 are included in the existing keyword words in the cloud server 320 When there are keywords, the existing standard sample library is directly used as the training sample library, and the deep learning algorithm is used for training to obtain a new scene-based keyword recognition model for the keywords; when part of the vocabulary in the keywords When not included in the existing keywords in the cloud server 320, the speech samples containing the part of the vocabulary are obtained from the Internet, and the standard sample database is expanded to form a training sample database, and then a new keyword recognition model is generated by training.

The user can also upload a voice sample of a specific person containing one or more words in the keyword through the video playback device 300 , for example, a student uploads a teacher's voice and audio data to the video playback device 300 . The video playback device 300 uploads the voice sample of the specific person to the cloud server 320, so as to expand the standard sample library of the cloud server 320, so that the cloud server 320 can base on the training sample library of the voice sample of the specific person that contains at least the keyword , and train to obtain a new keyword recognition model.

In response to the update request of the video playback device 300 , the cloud server 320 sends the scene-based keyword recognition model obtained by training to the video playback device 300 .

After receiving the keyword recognition model from the cloud server 320, the video playback device 300 downloads the keyword recognition model to the headset 310, so that the headset 310 updates its locally stored keyword recognition model.

It should be noted that the process of setting keywords and obtaining a new keyword recognition model can be completed in the initialization step, and can also be completed in each appropriate time during system operation. The specific process can be determined according to the actual situation. This does not limit.

Step 2, the earphone 310 receives the audio stream.

After completing the system initialization, the user can start to receive and play the audio stream from the video playback device 300 through the button 314 on the headset 310 . For example, students can watch the online course through the headset 310 and the tablet computer 300 at this time.

Step 3, the headset 310 acquires the voice signal in the audio stream, performs voice recognition on the voice signal, and uses the scene-based keyword recognition model to detect in real time whether the voice signal contains a preset keyword.

The headset 310 has a built-in speech recognition unit, which can be an embedded neural network processor, used to construct a neural network based on the keyword recognition model, and use a deep learning algorithm for data processing to perform real-time key on the continuously input speech signals. word recognition.

The audio stream of the online course may include various sound signals such as music and voice. The headset 310 extracts the voice signal, and uses a scene-based keyword recognition model and a deep learning algorithm to detect whether the voice signal contains preset keywords. For example, if the student presets the keyword "summary", when the teacher of the online class says "Let's summarize the main content of this class", the keyword can be detected and recognized in the current speech signal; If your name or student number is also used as a keyword, the headset 310 can play a good role as an auxiliary reminder when you are called by the online class teacher.

When the keyword is not identified, the earphone 310 continues to receive and play the audio stream, and does not enter into the execution of the following steps. It should be understood that, when the system performs the key information reminder, the process of receiving and playing the audio stream by the earphone 310 may not be affected.

Step 4, the headset 310 generates a key information reminder and records an audio stream.

When the headset 310 detects and recognizes that the preset keyword is contained in the current voice signal, it will make its vibrator start to vibrate. The user can stop the vibration of the earphone 310 by pressing the button 315 . If the vibration exceeds a predetermined vibration time, such as 10 seconds, and the user does not stop the vibration, the vibration may be automatically stopped, and the LED indicator 312 will start to flash red light. The red light can continue to flash for a longer flashing time, or it can continue to flash until the user presses the key 315 to stop it. When the earphone 310 detects that the current state of the LED indicator 312 is a working state (flashing red) before generating a new vibration, no new vibration is generated, but the current working state of the LED indicator 312 is maintained. In this way, if the student is still wearing the headset at this time, he can make him pay attention to the key information through vibration; and if he has taken off the headset, the light effect can be used to remind him.

The earphone 310 also starts to record the received audio stream while generating the key information reminder. Specifically:

Within the first predetermined period of time, the recorded audio stream is locally stored. The first predetermined duration should be less than or equal to the maximum duration that the headset 310 can store the audio stream. The first predetermined duration may be a preset fixed value. For example, the maximum duration of the audio stream that can be stored by the earphone 310 is 2 minutes, then the first predetermined duration may be 2 minutes, or the first predetermined duration may be 30 seconds, then the earphone The 310 can store up to 4 audio streams with a maximum duration of 30 seconds.

When the headset 310 starts to record the received audio stream, it also sends a recording start instruction and the detected keyword vocabulary to the video playback device 300 .

After receiving the recording start instruction sent by the headset 310, the video playback device 300 starts to record the sent audio stream.

Step 5: The video playback device 300 converts the recorded voice signal into text information and stores it.

The video playback device 300 can acquire the voice signal in the recorded audio stream, and convert the full text of the voice signal into text by using various voice-to-text methods in the prior art, and then store it. When storing, the received keyword vocabulary, words and recordings detected by the earphone 310 may also be stored in association, so that the user can select and refer to them later.

Step 6, the recording stops.

When the user inputs a recording stop instruction through the key 315, or when the duration of continuous recording exceeds the first predetermined duration but still does not receive the stop recording instruction from the user, the headset 310 will automatically stop recording the audio stream.

The video playback device 300 will automatically stop recording the audio stream when the user inputs a recording stop instruction through the key 315, or when the duration of continuous recording exceeds the second predetermined duration without receiving the stop recording instruction from the user.

Step 7, playback of the recording.

In this embodiment, the user can play back the recording on the earphone 310, and can also play the recording on the video playback device 300.

For example, when the student starts the local playback function by pressing the button 315 twice continuously, the headset 310 will play the audio stream from the video playback device 300 at the same time as the locally stored recorded audio stream. When playing, the two audio streams can be mixed and then played, or one of the two headsets 311 can be made to play one audio stream, and the other headset can play another audio stream.

Or, when the student starts the remote playback function by pressing the button 315 three times in a row, the headset 310 sends a playback request instruction to the video playback device 300, and after the video playback device 300 receives the playback request command, it sends the recorded audio stream to the video playback device 300. Headphones 310.

In addition, students can also directly input playback instructions on the video playback device 300 to play the recorded audio stream stored in the video playback device 300 .

The student can also specify to play the recorded audio stream on the video playback device 300 .

Step 8, check the text information.

In this step, the student can consult the text information corresponding to the recorded audio stream in the video playback device 300, so that the student can review and take notes according to the text information.

It can be seen from the descriptions of the above embodiments and typical application scenarios that the key information reminder method, system, and embedded audio playback device provided by the embodiments of the present invention realize the key information of continuous speech on a small, low-power embedded device Real-time detection, reminder and playback, easy to use, simple to operate, and widely applicable, and can effectively remind, save and review key information, reduce the loss of users missing key information, and increase user satisfaction with remote audio and video applications.

Those of ordinary skill in the art should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of the two, in order to clearly illustrate the hardware and software interchangeability, the components and steps of each example have been generally described in terms of functions in the above description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Persons of ordinary skill in the art may use different methods of implementing the described functionality for each particular application, but such implementations should not be considered beyond the scope of this application.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.

The specific embodiments described above further describe the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (15)

1. an embedded audio playback device, comprising a loudspeaker and a communication unit, is characterized in that: also comprise a control unit, a storage unit, a voice recognition unit and a reminder unit, The communication unit receives the audio stream from the remote end; The speech recognition unit includes a keyword recognition model unit, and the keyword recognition model unit is used to store a scene-based keyword recognition model; The keyword is associated with the application scenario, and includes a set of vocabulary that needs to be focused on in the application scenario, and one or more of the vocabulary is pre-specified by the user; The voice recognition unit extracts a voice signal from the audio stream, and uses the scene-based keyword recognition model to detect in real time whether the voice signal contains the keyword; The control unit is configured to start recording the received audio stream when a keyword is included in the voice signal, and control the reminder unit to output key information reminders; The storage unit is used to store the recorded audio stream; The speaker is used to play the audio stream, or to play back the recorded audio stream in response to a playback instruction. 2. The embedded audio playback device according to claim 1, wherein the scene-based keyword recognition model is based on a voice sample containing the keyword in advance, and/or a specific keyword for the keyword. The training sample library of human speech samples is obtained by training with deep learning algorithm; The control unit is further configured to download the scene-based keyword recognition model from a remote end through the communication unit. 3. The embedded audio playback device of claim 2, wherein the speech recognition unit further comprises a speech preprocessing unit for preprocessing the input audio stream to eliminate noise, background vocals, music sound, extract the voice signal; The speech recognition unit further includes a neural network processing unit for performing data processing on the speech signal or the speech signal processed by the speech preprocessing unit by using a deep learning algorithm based on the keyword recognition model, so as to perform data processing on the speech signal. Words appearing in the signal are reasoned and judged to determine whether they contain keyword words. 4. embedded audio playback device as claimed in claim 1, is characterized in that: described reminding unit is indicator light module, vibrator module, text message generation module, voice message generation module, music message generation module one or more of. 5. The embedded audio playback device of claim 1, further comprising an input unit for receiving a recording stop instruction and a playback instruction input by a user; When the control unit contains keywords in the voice signal, it starts to continuously compress and encode the received audio stream and store it locally; The control unit stops recording when receiving the recording stop instruction or when the continuous recording time exceeds the first predetermined duration; The control unit plays the locally stored recording audio stream when receiving the playback local audio instruction; The control unit is also used to send a recording start instruction to the far end when the voice signal contains a keyword, so that the far end starts to continuously record the sent audio stream, when the continuous recording time does not exceed the second predetermined time. When the duration is long and a stop recording instruction is received, a recording stop instruction is sent to the remote end; The control unit sends a playback request to the remote end, and receives and plays the recorded audio stream stored at the remote end when receiving the remote end audio playback instruction. 6 . The embedded audio playback device according to claim 1 , wherein the embedded audio playback device is an earphone or a speaker with a call function. 7 . 7. A key information reminder system, characterized in that: comprising an embedded audio playback device and a remote device, The remote device receives a user-defined keyword vocabulary, and/or a user-provided voice sample of a specific person that contains at least the keyword, so as to obtain a scene-based keyword recognition model; the keyword Associated with the application scenario, which contains a set of vocabulary that needs to be focused on in the application scenario; The scene-based keyword recognition model is obtained by training based on a training sample library containing speech samples for the keyword and/or speech samples of a specific person for the keyword in advance; The embedded audio playback device communicates with the remote device, receives an audio stream from the remote device, and plays it; The embedded audio playback device also obtains a voice signal from the audio stream, and uses a scene-based keyword recognition model to perform voice recognition on the voice signal, and detects in real time whether the voice signal contains keywords; When the voice signal contains a keyword, the embedded audio playback device generates a reminder of key information and starts recording the received audio stream; The embedded audio playback device plays the recorded audio stream in response to the playback instruction. 8. key information reminder system as claimed in claim 7, is characterized in that: also comprises cloud server, The remote device communicates with the cloud server, and sends the keyword and/or the voice sample of a specific person to the cloud server; The cloud server uses the received keywords and/or the voice samples of the specific person to expand its standard sample library to form a training sample library, and uses deep learning algorithm training to obtain the training sample library based on the training sample library. Scenario-based keyword recognition model; The remote device receives the scene-based keyword recognition model from the cloud server, and downloads the scene-based keyword recognition model to the embedded audio playback device. 9. The key information reminder system according to claim 7, wherein the remote device uses the keyword vocabulary input by the user and/or the voice sample of a specific person provided by the user that at least contains the keyword for the The standard sample library is expanded to form a training sample library, and based on the training sample library, a deep learning algorithm is used to train to obtain the scene-based keyword recognition model; The remote device downloads the scene-based keyword recognition model to the embedded audio playback device. 10. A key information reminder method, characterized in that: Receive a user-defined keyword vocabulary, and/or a user-provided voice sample of a specific person that contains at least the keyword; the keyword is associated with an application scenario, and includes a set of items that need to be focused on in the application scenario vocabulary; The scene-based keyword recognition model is obtained by training based on a training sample library comprising speech samples for the keyword and/or speech samples of a specific person for the keyword; When receiving and playing an audio stream, obtain a voice signal from the audio stream; Use the scene-based keyword recognition model to perform speech recognition on the voice signal, and detect in real time whether the voice signal contains keywords; When a keyword is included in the voice signal, a key information reminder is generated, and the received audio stream is started to be recorded; In response to the playback instruction, the recorded audio stream is played. 11. key information reminder method as claimed in claim 10, is characterized in that: Collect a wide range of speech samples in advance to form a standard sample library; Acquiring at least a voice sample containing the keyword according to the keyword; Expanding the speech samples containing the keywords and/or the speech samples of the specific person to the standard sample library to form a training sample library, and using deep learning algorithm training based on the training sample library to obtain the scene-based keyword recognition model. 12. The key information reminder method as claimed in claim 10, characterized in that: in the described step of acquiring the voice signal from the audio stream, it also includes a preprocessing step of eliminating noise, musical sound, and background vocals; The using the scene-based keyword recognition model to perform speech recognition on the speech signal or the preprocessed speech signal, and detecting in real time whether the speech signal contains keywords, specifically includes: constructing a recognition system based on the keyword The deep learning neural network of the model continuously inputs the speech signal into the deep learning neural network for data processing, so as to infer and judge the words appearing in the speech signal, and determine whether a keyword word is included in the speech signal. 13. The key information reminder method according to claim 10, wherein the recording of the received audio stream specifically comprises: when the voice signal contains a keyword, starting to continuously compress the received audio stream encoded and stored locally; When receiving a recording stop instruction or the continuous recording time exceeds the first predetermined duration, stop the local recording; The playing the recorded audio stream in response to the playback instruction specifically includes: in response to the playback local audio instruction, playing the locally stored recorded audio stream. 14. key information reminder method as claimed in claim 10, it is characterised in that: described recording the received audio stream, specifically comprising: when the voice signal contains a keyword, sending a recording start instruction to the far end, the far end Start continuous recording of the sent audio stream and store it remotely; When the continuous recording time does not exceed the second predetermined duration and a stop recording instruction is received, the recording stop instruction is sent to the remote end, and the remote end stops recording; The playing the recorded audio stream in response to the playback instruction specifically includes: in response to the playback remote audio instruction, sending a playback request to the remote end, and receiving and playing the recorded audio stream stored at the remote end. 15. The key information reminder method according to claim 10, wherein the key information reminder is a combination of one or more forms in a visual reminder, a tactile reminder and an auditory reminder; The visual reminder includes light effect reminder and remote text message reminder; The tactile reminder includes a vibration reminder; The auditory reminders include voice reminders and music reminders.

Images