KR102228017B1 - Stand-along Voice Recognition based Agent Module for Precise Motion Control of Robot and Autonomous Vehicles - Google Patents

Abstract

The present invention aims to provide a standalone embedded voice recognition technology capable of accurately recognizing and interpreting robot control voice commands even in a space without internet connection, and a voice recognition-based agent module capable of precisely controlling the motion of an autonomous vehicle. There is this.
To this end, the independent voice recognition-based agent module for precise motion control of robots and autonomous vehicles according to an embodiment of the present invention performs language processing to output the input voice as a recognizable text, and then converts the input voice into a speed control text for motion control. A voice recognition engine that converts and assigns; A robot operating system that receives a speed control command from the speech recognition engine through a programming language to which a speed control text is assigned and transmits a speed value to drive a robot or an autonomous vehicle; It may include a driving unit that receives the speed value from the robot operating system and drives the robot or autonomous moving object.

Description

Stand-along Voice Recognition based Agent Module for Precise Motion Control of Robot and Autonomous Vehicles}

The present invention relates to a standalone voice recognition-based agent module for precise motion control of robots and autonomous vehicles, and in particular, by performing voice command recognition and interpretation using a voice recognition algorithm including an acoustic model, a language model, and a data dictionary, The present invention relates to a single-type voice recognition-based agent module for precise motion control of robots and autonomous vehicles that can precisely control the motion of robots or autonomous vehicles.

In general, voice recognition refers to a technology that receives voice through a wired or wireless communication method such as a microphone and converts it into words or sentences. Since such voice recognition can add convenience to humans, it is widely used in the field of portable devices such as robots, mobile phones, smart homes, automobiles, and infotainment in recent years.

Accordingly, voice recognition has been steadily studied, and active research is expected to continue in the future.

Recently, an open source module for voice recognition has been released, and the use of voice recognition-based smart homes and personal assistants using open modules such as Amazon Alexa and Google Home is in the spotlight.

However, in the case of such an open source module, there is a disadvantage that the service is provided only through an Internet connection. Therefore, the stand along embedded voice recognition module can freely and precisely control the robot even in a space without an Internet connection. Development is in demand.

In order to solve the above problems, the present invention is a standalone embedded voice recognition technology capable of accurately recognizing and interpreting robot control voice commands even in a space without an Internet connection, and a voice recognition base capable of precisely controlling the motion of an autonomous vehicle. The purpose is to provide an agent module.

A standalone voice recognition-based agent module for precise motion control of robots and autonomous vehicles according to an embodiment of the present invention to solve the above problems, after language processing to output the input voice as a recognizable text, for motion control A voice recognition engine that converts and assigns speed control text; A robot operating system that receives a speed control command through a programming language to which a speed control text is assigned from the speech recognition engine and transmits a speed value to drive a robot or an autonomous vehicle; It may include a driving unit that receives the speed value from the robot operating system and drives the robot or autonomous moving object.

Here, the voice recognition engine may include a Pocketsphinx or other open voice recognition module.

In addition, the speech recognition engine includes a pre-processing unit for extracting a feature vector required for speech recognition and a speech recognition unit for storing a speech recognition algorithm, and analyzing the feature vector extracted from the pre-processing unit through the speech recognition algorithm to process language. can do.

In addition, the speech recognition algorithm may include an acoustic model that allows adaptability to a voice input through a microphone; A language model for converting the extracted feature vector into a recognizable text format by comparing the extracted feature vector with the adaptive acoustic model, and the language model can be converted into a recognizable text format when comparing the extracted feature vector and the acoustic model. It can include a data dictionary to determine if it is there.

In addition, the programming language may be Python or C/C++.

In addition, transmission of the speed control command from the programming language to the robot operating system may be transmitted through a wired or wireless communication method.

In addition, the driving unit may include a low-level processor for receiving a speed value from the robot operating system and inputting a speed as a low-level signal; It may include a PWM generator for pulse-modulating the speed signal input through the low-level processor, and a DC motor for driving a robot or an autonomous vehicle according to the pulse modulated from the PWM generator.

The independent voice recognition-based agent module for precise motion control of robots and autonomous vehicles according to an embodiment of the present invention has a feature that can freely and precisely control the robot even in a space without an Internet connection.

In addition, by using the speaker adaptation technique of Maximum Likelihood Linear Regression (MLLR) and Maximum A Posteriori (MAP), the acoustic model, which is a component of the present invention, can perform more accurate speech recognition.

In addition, by using an open speech recognition engine including Pocketsphinx, an open speech recognition engine, it can be provided at a relatively low price.

1 is a block diagram showing the configuration of a single-type voice recognition-based agent module for precise motion control of robots and autonomous vehicles according to an embodiment of the present invention.
2 is a block diagram showing the configuration of a speech recognition engine, which is one configuration of the present invention.
3 is a block diagram showing a configuration of a recognition unit, which is a component of the speech recognition engine of FIG. 2.
4 is a block diagram showing a speaker adaptation step of an acoustic model, which is a component of the present invention.
5 is a flowchart illustrating an operation of a single-type voice recognition-based agent module for precise motion control of robots and autonomous vehicles according to an embodiment of the present invention.
6 is a flowchart of the operation of step (c) of FIG. 5.
7 is a flowchart of the operation of step (f) of FIG. 5.

Hereinafter, the description of the present invention with reference to the drawings is not limited to a specific embodiment, and various transformations may be applied and various embodiments may be provided. In addition, the content described below should be understood to include all conversions, equivalents, and substitutes included in the spirit and scope of the present invention.

In the following description, terms such as first and second are terms used to describe various elements, and their meanings are not limited, and are used only for the purpose of distinguishing one element from other elements.

The same reference numbers used throughout this specification denote the same elements.

The singular expression used in the present invention includes a plurality of expressions unless the context clearly indicates otherwise. In addition, terms such as "comprise", "include" or "have" described below are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification. It is to be construed and not to preclude the possibility of the presence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 1 to 7.

1 is a block diagram showing the configuration of a single-type voice recognition-based agent module for precise motion control of robots and autonomous vehicles according to an embodiment of the present invention. FIG. 3 is a block diagram showing a configuration of a recognition unit, which is a component of the speech recognition engine of FIG. 2, and FIG. 4 is a block diagram showing a speaker adaptation step of an acoustic model, which is a component of the present invention. .

First, referring to Figs. 1 to 4, a single-type voice recognition-based agent module for precise motion control of a robot and an autonomous mobile according to an embodiment of the present invention is installed in a robot or an autonomous mobile body, and a voice recognition engine 10, It may include a robot operating system 30, a driving unit 40.

Specifically, the voice recognition engine 10 may recognize a voice input from a microphone installed in a robot or an autonomous mobile body and a voice transmission device based on wired/wireless communication. In addition, the voice recognition engine 10 may be composed of Pocketsphinx.

This is an open speech recognition engine, which has an inexpensive advantage, has high compatibility with a robot operating system (ROS) configured to control a robot or autonomous vehicle, programming languages such as C/C++ or Python, and high recognition speed. have. However, the pocketshpinx is a preferred example and is not limited and may be provided with another voice recognition engine 10.

Hereinafter, a description will be made on the basis of a voice recognition engine 10 composed of a pocketshpinx, which is a preferred example.

The voice recognition engine 10 can perform language processing to output the voice input from the installed microphone and the wired/wireless communication-based voice transmission device as a recognizable text, and convert the language-processed text into speed control text for motion control. Can be assigned.

To this end, the voice recognition engine 10 may include a preprocessor 12 and a recognition unit 14 as shown in FIG. 2.

The preprocessor 12 may extract feature vectors required for speech recognition. That is, when a voice is input from a microphone and enters the voice recognition engine 10, the preprocessor 12 may extract a feature vector capable of expressing a phonetic feature well from the voice. In this case, the preprocessor 12 may extract a feature vector in units of 1/100 (second).

In addition, the preprocessor 12 may extract the feature vector using a Mel Frequency Cepstral Coefficients (MFCC) algorithm when extracting the feature vector. Here, the MFCC algorithm may extract features by dividing the entire input sound into a short time, and analyzing a spectrum for this period. For example, this is a method of dividing the length of a certain section into units of 20 to 40 ms and calculating a spectrum corresponding to each unit, that is, a frequency.

Meanwhile, the preprocessor 12 may extract a feature vector by performing noise processing for processing noise including channel distortion and background noise originating from a microphone, a line, and the like. In order to increase the success rate of speech recognition, a method such as compensating after extracting a feature vector or introducing a feature vector that is strong against noise can be used.

Through this, the preprocessor 12 from which the feature vector has been extracted may transmit the feature vector to the recognition unit 14.

The recognition unit 14 may perform language processing by pattern analysis of the feature vector extracted from the preprocessor 12. In addition, the recognition unit 14 may store the speech recognition algorithm 16 and may perform language processing through the speech recognition algorithm 16. Here, the speech recognition algorithm 16 may include an acoustic model 16a, a language model 16b, and a data dictionary 16c, as shown in FIG. 3.

More specifically, the recognition unit 14 may receive the feature vector of the speech extracted from the preprocessor 12 and compare the pattern with the acoustic model 16a stored in the database of the speech recognition engine 10 to obtain a recognition result. . Here, as shown in FIG. 4, the acoustic model 16a is formed to have adaptability to the speaker's voice transmitted through the speaker in order to increase the recognition rate, and for this purpose, the acoustic model 16a is MLLR (Maximum Likelihood Linear Regression) and A MAP (Maximum A Posteriori) adaptation technique can be used.

At this time, the acoustic model 16a can execute MAP (Maximum A Posteriori) after adapting MLLR (Maximum Likelihood Linear Regression), and through this, the acoustic model 16a provides a sample that is as close as possible to the speaker's voice, thereby providing a speech recognition engine. (10) This can make it possible to accurately recognize the voice.

Meanwhile, the acoustic model 16a may be configured in Korean or English.

The language model 16b may perform language processing on the voice recognized through the acoustic model 16a. To this end, the language model 16b may include a word unit search and a sentence unit search.

The word-by-word search is performed by including phonemes, and possible candidate words or candidate phonemes may be extracted through a word-based or phoneme-based pattern comparison with the acoustic model 16a stored in the database. In this case, the candidate word or candidate phoneme through the above process may be searched for each sentence.

The sentence-by-sentence search can determine the most suitable word or phoneme by determining whether it conforms to a grammar structure, sentence context, or a specific subject using a data dictionary based on information on candidate words or candidate phonemes.

For example, in the sentence'We go to the beach', suppose it is difficult to distinguish between'Eun' and'Nung' due to an indefinite pronunciation. Can be produced as a result. In this case, in the sentence-by-sentence search, through the sentence structure analysis using the data dictionary 16c,'A' plays a role of investigation in the sentence, but it is recognized that there is no investigation'capability' and can be excluded from the candidate.

That is, the language model 16b may perform a language processing process to improve recognition performance by restricting vocabulary and grammar structures. Through this method, the speech recognition of the speech recognition engine 10 is performed faster, and the speech recognition result may be more accurate.

Here, the language model 16b is based on statistical pattern recognition, and a Hidden Markov Model (HMM) technique, which is a method in which a word-based search and a sentence-based search process are integrated into one, may be used. This is the most suitable speech unit for the unknown pattern by storing statistical information of the patterns corresponding to the speech unit in the form of a probability model, and calculating the probability that the unknown pattern from each model can come out when an unknown input pattern comes in. It's a way to find out.

On the other hand, the language model 16b may transmit the processed language as text so as to control the motion of a robot or an autonomous vehicle during language processing.

That is, the voice recognized by comparing the feature vector of the voice extracted by the preprocessor 12 with the acoustic model 16a adapted to the speaker's voice extracts candidate words or candidate phonemes through the language model 16b, Candidate words or candidate phonemes are identified based on the data dictionary (16c) to determine the most appropriate word or phoneme, and are divided into correct sentence units. At this time, the speech processed to be divided into sentences will be converted into text through the language model (16b) I can.

As described above, the language-processed text may be converted and assigned to a speed control text capable of adjusting the speed of a motor for motion control of a robot or an autonomous vehicle through the voice recognition engine 10 as described above.

The assigned speed control text may be replaced with a robot control command through the programming language 20 to control motion and transmitted to the robot operating system 30.

In this case, the programming language 20 may be C/C++ or Python. In addition, the programming language 20 may be connected to the robot operating system 30 by wire or wirelessly. That is, the programming language 20 assigned the speed control text through the voice recognition engine 10 may generate a robot control command through an algorithm and transmit it to the robot operating system 30 in a wired or wireless manner. Here, the wireless method may be configured by a method such as Bluetooth or Wi-Fi.

The robot operating system 30 may receive a speed control command from the programming language 20. In addition, the robot operating system 30 may control the driving unit 40 by transmitting a speed value to the driving unit 40 that drives the robot or the autonomous vehicle. That is, the robot operating system 30 may control the driving unit 40 to control the motion of the robot or the autonomous moving object by adjusting the speed value according to the speed control command from the programming language 20.

The driving unit 40 may receive a speed value from the robot operating system 30 to drive a robot or an autonomous moving object. To this end, the driving unit 40 may include a low-level processor 42, a PWM generator 44, and a DC motor 46.

Specifically, the low-level processor 42 may program and input a speed value transmitted through the robot operating system 30 as a low-level signal.

The PWM generator 44 may be programmed from the low-level processor 42 to a low-level to modulate a pulse of an input low-level signal. That is, the speed value can be controlled through pulse modulation of the low-level signal expressing the speed value.

The DC motor 46 may be connected to a robot or a wheel of an autonomous vehicle, and may drive the robot or autonomous vehicle according to a pulse modulated by the PWM generator 44. At this time, it is preferable that the DC motor 46 is provided for each wheel, and the motion of the robot or the autonomous vehicle may be controlled by varying the control speed of each DC motor 46.

Hereinafter, a method of operating a single-type voice recognition-based agent module for precise motion control of robots and autonomous vehicles according to an embodiment of the present invention will be described with reference to FIGS. 5 to 7.

5 is a flowchart illustrating an operation of a single-type voice recognition-based agent module for precise motion control of a robot and an autonomous mobile according to an embodiment of the present invention, and FIG. 6 is an operation flowchart of step (c) of FIG. 5, and FIG. 7 is This is the operation flow chart of step 5(f).

Referring to FIGS. 5 to 7, a method of operating a single-type voice recognition-based agent module for precise motion control of robots and autonomous vehicles according to an embodiment of the present invention may proceed in the following steps (a) to (f). .

(a) User's voice input step through a microphone (S100)

-The user can input voice commands through the microphone installed on the robot or autonomous vehicle. At this time, the voice command may be transmitted to the voice recognition engine 10.

(b) extracting a feature vector of the input speech (S200)

-When the voice command reaches the voice recognition engine 10, the preprocessor 12 of the voice recognition engine 10 may extract a feature vector of the input voice. Here, the speech recognition engine 10 may be provided with Pocketsphinx, and feature vectors may be extracted in units of 1/100 (second) using a Mel Frequency Cepstral Coefficients (MFCC) algorithm.

(c) converting the feature vector into recognizable text using a speech recognition algorithm (S300)

-This step can be performed in the recognition unit 14 of the voice recognition engine 10. In this case, the speech recognition algorithm 16 may be stored in the recognition unit 14. That is, the recognition unit 14 may receive the feature vector extracted from the preprocessor 12, and the recognition unit 14 may convert the text into a recognizable text using the stored speech recognition algorithm 16.

Specifically, the speech recognition algorithm 16 may include an acoustic model 16a, a language model 16b, and a data dictionary 16c, and may be performed in the following three steps.

Step 1: Step of adapting the acoustic model 16a to the speaker's voice (S310)

-The acoustic model 16a can be compared with the feature vector extracted from the preprocessing unit 12 to derive the speech recognition result.At this time, the acoustic model 16a can adapt the speaker in consideration of the differences in phonetic characteristics for each speaker. It can be formed to be. Here, the acoustic model 16a can use the MLLR (Maximum Likelihood Linear Regression) and MAP (Maximum A Posteriori) adaptation techniques, and after performing the MLLR (Maximum Likelihood Linear Regression) adaptation, MAP (Maximum A Posteriori) adaptation is sequentially performed. You can proceed.

Through this, the recognition rate can be further increased.

Step 2: Comparing the acoustic model adapted to the speaker and the feature vector to recognize speech (S320)

-In step 1, if the initial acoustic model 16a is adapted to the speaker, the feature vector can be compared with the adaptive acoustic model 16a. In this case, the acoustic model 16a may perform more accurate recognition through speaker adaptation.

Step 3: After the language model 16b extracts a candidate phoneme or a candidate word according to the recognized voice, it determines the correct voice using the data dictionary 16c, and converts it into a recognizable text form for motion control of robots and autonomous vehicles. Converting step (S330)

-According to the voice recognized in step 2, the language model 16b may extract a candidate phoneme or a candidate word through a word unit search and a sentence unit search through HMM (Hidden Markov Model) technique. Here, the language model 16b may determine the most suitable word or phoneme by comparing the extracted candidate phoneme or candidate word through the preset data dictionary 16c.

The sentence determined through the above process may be converted into a recognizable text format for precise motion control of robots and autonomous vehicles.

(d) converting the text converted to be recognizable into speed control text (S400)

-Recognizable text derived through the speech recognition algorithm 16 may be converted into text capable of controlling the speed and transmitted to the programming language 20. This step may be performed in the voice recognition engine 10.

(e) generating a speed control command through a programming language for the speed control text (S500)

-The speed control text delivered to the programming language 20 can generate a speed control command through a coded algorithm. At this time, the generated speed control command may be transmitted to a robot operating system that controls a robot or an autonomous mobile body in a wired or wireless method such as Wi-fi or Bluetooth.

Meanwhile, the programming language 20 may be provided in C/C++ or Python.

(f) step of controlling the driving unit 40 for operating the motion of the robot or autonomous vehicle by the robot operating system 30 according to the speed control command (S600)

-The robot operating system 30 that has received the speed control command can be controlled by sending a speed command (speed value) so that the driving unit 40 that operates the motion of the robot or the autonomous moving object generates a speed.

Here, the driving unit 40 may perform the following three steps including the low-level processor 42, the PWM generator 44, and the DC motor 46.

Step 1: Step of receiving a speed value (speed command) by the low-level processor 42 from the robot operating system 30 and inputting the speed as a low-level signal (S610)

-The low-level processor 42 may receive a speed value from the robot operating system 30. In this case, the low-level processor 42 receiving the speed value may input the speed as a low-level signal through programming.

Step 2: The PWM generator pulse-modulating the low-level speed signal input through the low-level processor (42) (S620)

-The PWM generator 44 receives the low-level speed signal programmed through the low-level processor 42 and pulse-modulates the signal.

Step 3: Driving the DC motor to operate the motion of the robot or autonomous vehicle according to the pulse modulated from the PWM generator 44 (S630)

-The pulse modulated from the PWM generator 44 is transmitted to each DC motor 46 provided for each wheel of the robot or autonomous vehicle, and the speed of the DC motor 46 is controlled according to the pulse, so that the motion of the robot and the autonomous vehicle is controlled. Can be controlled.

Accordingly, the conventional open voice recognition module was provided with a service only through an Internet connection, but the standalone voice recognition-based agent module for precise motion control of robots and autonomous vehicles according to an embodiment of the present invention and its operation method are not connected to the Internet. It has the feature of being able to freely control the robot even in a space without it.

The embodiments of the present invention have been described above with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. You will be able to understand. Accordingly, the embodiments described above are illustrative and non-limiting in all respects.

10: voice recognition engine
12: pre-treatment unit
14: recognition unit
16: speech recognition algorithm
16a: acoustic model
16b: language model
16c: data dictionary
20: programming language
30: Robot operation system
40: drive unit
42: low-level processor
44: PWM generator
46: DC motor

Claims (2)

As a standalone embedded voice recognition-based agent module that can accurately recognize and interpret robot control voice commands even in a space without internet connection,
A speech recognition engine that performs language processing to output a speech input through a microphone installed in a robot or an autonomous mobile body and a speech transmission device based on wired/wireless communication as a recognizable text, and converts and assigns it to a speed control text for motion control;
A robot operating system that receives a speed control command through a programming language to which a speed control text is assigned from the speech recognition engine, and transmits and controls a speed value to drive the robot or autonomous vehicle;
It includes a driving unit that receives a speed value from the robot operating system and drives a robot or an autonomous moving body,
The voice recognition engine,
Feature vectors required for speech recognition are compensated for noise including channel distortion and background noise after feature vectors are extracted, or noise is processed by introducing feature vectors that are strong against noise, and sound input using MFCC (Mel frequency cepstral coefficients) algorithm A preprocessor that divides the entire short time, analyzes the spectrum for this section, and extracts it in units of 1/100 (second), and
A recognition unit storing a speech recognition algorithm, and analyzing a feature vector extracted from the preprocessing unit through the speech recognition algorithm to process a language,
The speech recognition algorithm,
After adaptation of MLLR (Maximum Likelihood Linear Regression) to the voice input through the microphone, the MAP (Maximum A Posteriori) adaptation technique is used to have the speaker's voice and adaptability, and an acoustic model consisting of Korean or English;
Converts the extracted feature vector into a recognizable text form after pattern comparison with the adaptive acoustic model, including word-by-word and sentence-by-sentence searches, and language processing by constraining vocabulary and grammar structures to convert the processed language into text Based on the statistical pattern recognition of HMM (Hidden Markov Model), statistical information of patterns corresponding to speech units is stored in the form of a probability model, and when an unknown input pattern comes in, an unknown pattern from each model comes out. A language model that finds the most suitable phonetic unit for an unknown pattern by calculating the possible probability and
A data dictionary for determining whether the language model can be converted into a recognizable text form when comparing the extracted feature vector and the acoustic model,
The recognition unit receives the feature vector of the speech extracted from the preprocessor and compares the pattern with the acoustic model to obtain a recognition result,
The word-by-word search of the language model is performed including a phoneme-by-phone search, and after extracting a possible candidate word or a candidate phoneme through a word-by-word or phoneme-by-phone pattern comparison with the acoustic model stored in the database, the sentence-by-sentence search is performed. ,
The sentence-by-sentence search determines the most suitable word or phoneme by determining whether it conforms to a grammar structure, sentence context, and a specific topic using a data dictionary, based on information on candidate words or candidate phonemes,
The voice recognition engine is composed of Pocketsphinx,
The programming language is Python or C/C++,
Transmission of the speed control command from the programming language to the robot operating system is transmitted through a wired communication method,
The driving unit,
A low-level processor for receiving a speed value from the robot operating system and inputting a speed as a low-level signal;
PWM generator for pulse-modulating the speed signal input through the low-level processor and
A standalone voice recognition-based agent module for precise motion control of robots and autonomous vehicles, including a DC motor that drives a robot or an autonomous vehicle according to a pulse modulated from the PWM generator. delete

Images