KR20190126552A - System and method for providing information for emotional status of pet - Google Patents

Abstract

According to the present invention, a system for providing information for an emotional state of a pet comprises: a client to collect voice information of a pet to transmit the voice information; and an interpretation server to receive the collected voice information from the client, and analyze emotional state information of the pet based on the received voice information to transmit the emotional state information to the client. The interpretation server includes: a communication unit to receive the voice information transmitted from the client; a feature vector extraction unit to extract a voice feature vector from the received voice information; an emotional state analysis unit to perform machine learning using the voice feature vector extracted by the feature vector extraction unit as input data to extract emotional state information corresponding to the pet; a control unit to control operations of the communication unit, the feature vector extraction unit, the emotional state analysis unit, and a state information output unit; and a storage unit to store data for operations of the feature vector extraction unit, the emotional state analysis unit, the state information output unit, and the control unit. The communication unit transmits the emotional state information analyzed by the emotional state analysis unit to the client in accordance with control of the control unit.

Description

System and method for providing information about emotional status of pets {System and method for providing information for emotional status of pet}

The present invention relates to a technique for communicating with a companion animal, and more particularly, to a technology for providing a companion animal by analyzing an emotional state.

In recent years, as researches on the companion animal's voice information (speech) have been conducted to find out the emotions and conditions of the companion animal, devices and programs for analyzing the animal's sound have been introduced.

At this time, the method of analyzing the sound of the companion animal is made by analyzing the frequency. In this regard, when it is determined that the frequency characteristics are determined from the sound of the companion animal and determined to be in a predetermined preset state, video call connection is automatically made to the user terminal, and the user image information is output on the screen of the electronic device or the display unit. Disclosed is a technique for providing a color spectrum image that changes at predetermined intervals around information.

However, even if you analyze the sound of the pets, it is only to grasp the simple emotions, it is only to satisfy the curiosity because no accurate analysis method is proposed.

In particular, when analyzing the frequency characteristics from the voice information of the companion animal, it is difficult to accurately analyze when there is a sound due to the overlapping emotion. In addition, in order to accurately analyze emotions, a process of normalizing frequencies or sounds that vary according to types, ages, and body information as well as frequencies is required. Since such processes are not included, accurate analysis is not easy.

The problem to be solved by the present invention is to receive the barking sound of the pet, that is, voice information (speech), it is possible to grasp the emotional state of the companion animal through repetitive learning through the machine learning, it can be provided to each client The present invention relates to a system and method for providing information on the emotional state of a companion animal.

The system for providing information on the emotional state of the companion animal according to the present invention for solving the problem is to collect and transmit the voice information of the companion animal; And an interpreter server for receiving the collected voice information from the client and analyzing and transmitting the emotional state information of the companion animal to the client based on the received voice information. A communication unit for receiving the transmitted voice information; A feature vector extracting unit extracting a voice feature vector from the received voice information; An emotional state analyzing unit configured to extract the emotional state information corresponding to the companion animal by performing a machine learning using the voice feature vector extracted by the feature vector extracting unit as input data; A controller for controlling operations of the communication unit, the feature vector extracting unit, the emotional state analyzing unit, and the state information output unit; And a storage unit which stores data for operations of the feature vector extractor, the emotion state analyzer, the state information output unit, and the controller, wherein the communication unit analyzes the emotion state analyzer under control of the controller. The emotion state information is transmitted to the client.

The feature vector extractor extracts the speech feature vector using any one of an auto-encoder and a Mel Frequency Cepstral Coefficient (MFCC).

The emotional state analysis unit may extract the emotional state information using any one of a machine learning algorithm of a deep neural network (DNN) algorithm and a recurrent neural network (RNN) algorithm.

The communication unit may transmit the emotion state information to the client through wired communication or wireless communication.

The method for providing information on the emotional state of the companion animal according to the present invention for solving the above problems comprises the steps of collecting the voice information of the companion animal to transmit to the interpretation server; Extracting, by the interpreter server, a speech feature vector from the speech information transmitted from the client; Extracting, by the interpreter server, emotional state information corresponding to the companion animal by performing machine learning using the extracted voice feature vector as input data; And transmitting, by the interpreter server, the extracted emotional state information to the client. And outputting, by the client, the received emotional state information.

The extracting of the speech feature vector may include extracting the speech feature vector using any one of an auto-encoder and a Mel Frequency Cepstral Coefficient (MFCC).

The extracting of the emotional state information may include extracting the emotional state information using any one of a machine learning algorithm of a deep neural network (DNN) algorithm and a recurrent neural network (RNN) algorithm.

The transmitting of the emotional state information may include transmitting the emotional state information to the client through wired communication or wireless communication.

According to the present invention, by using the voice information (speech) of the companion animal as input data, the emotional state of the companion animal is identified through repeated machine learning and provided to each client, whereby the guardian of the companion animal has his or her companion animal. Can accurately recognize the emotional state of the, and can pinpoint the current state of the pet.

In addition, the caregiver of the pet can correctly recognize the emotional state of the pet, it is possible to present an appropriate command to the pet to allow the pet to induce the correct behavior or response in response to the recognized emotional state.

1 is a system block diagram of an embodiment for explaining a system for providing information on an emotional state of a companion animal according to the present invention.
FIG. 2 is a block diagram illustrating an exemplary embodiment for describing the detailed components of the interpreter server shown in FIG. 1.
3 is a reference diagram illustrating a basic operation structure of an auto-encoder method corresponding to a feature vector extracting unit.
4 is a reference diagram illustrating an extraction of a speech feature vector by a Mel Frequency Cepstral Coefficient (MFCC) method corresponding to a feature vector extractor.
5 is a reference diagram illustrating a basic operation structure of a Recurrent Neural Network (RNN) algorithm corresponding to an emotional state analyzer.
6 is a reference diagram illustrating a basic operation structure of a deep neural network (DNN) algorithm corresponding to an emotional state analyzer.
7 is a reference diagram illustrating that the emotion state analysis unit extracts the emotion state information corresponding to the voice information transmitted from the client from various emotion state information of the pet stored in the storage unit.
8 is a flowchart of an embodiment for explaining a method for providing information on an emotional state of a companion animal according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention are provided to more fully explain the present invention to those skilled in the art, and the following embodiments may be modified in many different forms, the scope of the present invention It is not limited to the following embodiments. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a", "an" and "the" may include the plural forms as well, unless the context clearly indicates otherwise. Also, as used herein, "comprise" and / or "comprising" specifies the presence of the mentioned shapes, numbers, steps, actions, members, elements and / or groups of these. It is not intended to exclude the presence or the addition of one or more other shapes, numbers, acts, members, elements and / or groups. As used herein, the term "and / or" includes any and all combinations of one or more of the listed items.

Although the terms first, second, etc. are used herein to describe various members, regions, and / or portions, it is obvious that these members, components, regions, layers, and / or portions should not be limited by these terms. Do. These terms do not imply any particular order, up or down, or superiority, and are only used to distinguish one member, region or region from another member, region or region. Accordingly, the first member, region, or region described below may refer to the second member, region, or region without departing from the teachings of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings schematically showing embodiments of the present invention. In the drawings, for example, variations in the shape shown may be expected, depending on manufacturing techniques and / or tolerances. Accordingly, embodiments of the present invention should not be construed as limited to the specific shapes of the regions shown herein, but should include, for example, changes in shape resulting from manufacturing.

1 is a system block diagram of an embodiment for explaining the information providing system 10 for the emotional state of a companion animal according to the present invention.

Referring to FIG. 1, the information providing system 10 includes a client 100 and an interpreter server 200.

The client 100 collects the voice information of the companion animal and transmits it to the interpreter server 200.

The client 100 continuously detects the voice information of the pet, that is, the barking sound or the speech. To this end, the client 100 may include a microphone for collecting voice information. The client 100 arranges the continuously detected sounds in chronological order, and includes the first detected sound (sound1), the time interval (delay1) of the first detected sound and the second detected sound, and the second detected sound ( sound2), the time interval delay2 of the second and third detected sounds, and the third detected sound (sound3) may be continuously stored in the form of a vector up to an nth order. Since the client 100 analyzes the emotion of the companion animal in real time by detecting the companion animal's sound from the continuous barking of the pet, information for analyzing the emotion of the companion animal more accurately than simply analyzing the emotion with a single barking sound. Can be collected.

The client 100 converts the input analog signal into a digital signal in order to detect sound information of the pet's barking sound, and then detects the sound information, which is the time interval between the barking sound and the sound interval, from the converted digital signal. . Thereafter, the client 100 transmits the collected voice information to the interpretation server 200 using a communication network (not shown).

The communication network includes a wired communication network or a wireless communication network for transmitting and receiving data between the client 100 and the interpretation server 200. Thereafter, when the communication network receives the emotional state information about the companion animal from the interpreter server 200, it transmits it to the client 100.

The client 100 outputs the emotional state information received through the communication network. The client 100 may output the emotion state information as one of an audio signal and an image signal. The client 100 may output the emotional motion, the dialogue sentence, and the dialogue voice of the companion animal as the emotion state information. The client 100 may continuously play a 2D image file form to produce a 3D motion effect or drive a 3D motion animation file to generate a motion.

The client 100 is interpreted as a human conversation by the current pet's emotion and the current pet's setting environment, and the method of interpreting the current pet's type (Species), sex (Sex), age (Age) and interest You can use the dialogue and the dialogue voice that set up the dialogue according to conditions such as (Interest) and Personality (Character).

The status information output unit 140 may generate a conversation of a person based on rules based on other conditions such as current time or weather, in addition to the pet's emotion and the current pet's setting environment.

The interpretation server 200 receives the voice information provided from the client 100, analyzes the emotional state information of the companion animal based on the received voice information, and transmits it to the client 100.

FIG. 2 is a block diagram illustrating an exemplary embodiment for describing the detailed components of the interpreter server 200 illustrated in FIG. 1.

Referring to FIG. 2, the emotion state analysis device 200 of the companion animal includes a communication unit 210, a feature vector extractor 220, an emotion state analysis unit 230, a user interface unit 240, a controller 250, The storage unit 260 may be included.

The communication unit 210 may receive voice information provided from the client 100. To this end, the communication unit may include a wired communication module or a wireless communication module that can be connected to a wired communication network or a wireless communication network. The wired communication module may be a module that supports wired Internet, LAN, or the like. In addition, the wireless communication module may be a wireless Internet such as wireless fidelity (WiFi), a mobile Internet such as wireless broadband internet (WiBro) or world interoperability for microwave access (WiMax), a global system for mobile communication (GSM) or code division multiple (CDMA). 2G mobile networks such as access, wideband code division multiple access (WCDMA) or 3G mobile networks such as CDMA2000, 3.5G mobile networks such as high speed downlink packet access (HSDPA) or high speed uplink packet access (HSUPA), LTE ( 4G mobile communication network such as long term evolution) or LTE-Advanced network, and 5G mobile communication network.

The feature vector extractor 220 extracts a voice feature vector from the voice information received from the communication unit 210. The feature vector extractor 220 extracts a feature of the actual valid sound from the noise and the background sound in the input signal.

The feature vector extractor 220 may extract the speech feature vector using any one of an auto-encoder method and a Mel Frequency Cepstral Coefficient (MFCC) method.

The auto-encoder method is an algorithm that derives important features of the data in the process of outputting the same result as the input value. 3 is a reference diagram illustrating a basic operation structure of an auto-encoder method corresponding to the feature vector extractor 220.

The MFCC (Mel Frequency Cepstral Coefficient) method is a technique for extracting a feature of a sound. The MFCC is a technique for extracting a feature by dividing it into a short time of the input sound and analyzing the spectrum of the section. 4 is a reference diagram illustrating an extraction of a speech feature vector by a Mel Frequency Cepstral Coefficient (MFCC) method corresponding to the feature vector extractor 220.

The feature vector extractor 220 may perform the following process to perform the MFCC. For example, the feature vector extractor 220 cuts the sound signal of the input time domain into a small size frame. Thereafter, the feature vector extractor 220 calculates a Periodogram estimate (Periodogram Spectral Estimate) of the Power Spectrum for each frame. Thereafter, the feature vector extractor 220 applies a Mel filter bank to the calculated power spectrum, and adds energy to each filter. Thereafter, the feature vector extracting unit 220 takes a DCT after taking a log of all the? Filter bank energies. After that, the feature vector extractor 220 leaves only Coefficients 2 to 13 at the DCT value and discards the rest.

The feature vector extractor 220 continuously displays the companion animal's sound in frequency form (sound waveform) when the companion animal's barking sound is heard. In this case, the feature vector extractor 220 may extract the speech feature vector from the storage unit 270 using reference speech feature vector information corresponding to the emotion.

Each time the pet barks, the input waveform is almost parabolic. However, it is practically impossible to distinguish the pet's feelings just by barking once. In order to properly distinguish the emotions of the pets, it is necessary to grasp the time interval (sound information) between the barking sounds and the barking sounds of the pets, and the voice of the pets provided from the client 100 is composed of the following vectors. Since the voice vector may be represented by a different vector even with the same input sound according to the end point detection condition of the voice information, the same end point detection condition should be applied as much as possible. Samples tested in must be provided. In the form of a vector, the pet can bark once in the form of a vector value of zero crossing rate (ZCR) and energy per frame.

The emotional state analyzing unit 230 performs machine learning using the voice feature vector extracted from the feature vector extracting unit 220 as input data to extract emotional state information corresponding to the companion animal.

The emotional state analyzer 230 may extract the emotional state information using any one of a machine learning algorithm of a deep neural network (DNN) algorithm and a recurrent neural network (RNN) algorithm.

The Recurrent Neural Network (RNN) algorithm receives continuous data such as voice and outputs the result value considering the previous input value. 5 is a reference diagram illustrating a basic operation structure of a recurrent neural network (RNN) algorithm corresponding to an emotional state analyzer 230.

The deep neural network (DNN) algorithm corresponds to a deep neural network, which means an artificial neural network (ANN) including multiple hidden layers between an input layer and an output layer. do. 6 is a reference diagram illustrating a basic operation structure of a deep neural network (DNN) algorithm corresponding to the emotional state analyzer 230.

Deep neural networks can learn a variety of nonlinear relationships, including multiple hidden layers. Deep neural networks include Deep Belief Network (DBN), deep autoencoder (DBN) based on unsupervised learning, and synthesis for processing two-dimensional data such as images. Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) for time series data processing. Therefore, based on the data, the input (feature vector of extracted speech) and the output (emotion state) are determined and learned. As it is learned, weight values are determined for each connection between layers, such as between input layer and hidden layer 1, and after completion, the result can be output when other data is inputted.

In addition, the emotion state analyzer 230 may infer the emotion by comparing the voice feature vector detected by the feature vector extractor 220 with the pattern of the voice feature vector for each emotion stored in the storage unit 270. To this end, the emotional state analysis unit 230 uses the pattern comparison algorithms such as Dynamic Time Warping (DTW), Hidden Marcov Model (HMM), Artificial Neural Network (ANN), Fuzzy, or two or more of the most similar emotions. By judging the emotional state information for the companion animal can be recognized.

7 is a reference diagram illustrating that the emotion state analysis unit 230 extracts the emotion state information corresponding to the voice information transmitted from the client 100 among various emotion state information of the companion animal stored in the storage unit 260. . Referring to FIG. 7, the emotion state analyzer 230 may request emotions of a companion animal, demand joy, curiosity, loneliness, warning, angry, fear, and fear. Inferred from emotions such as (Fear). To this end, the emotion state analysis unit 230 may be connected to the storage unit 260 in which the above-described emotion feature vector for each emotion is stored.

The user interface 240 is for providing setting information about the companion animals, and includes a configuration button (for example, Species, Sex, Age, Interest, and Personality). Press (S2000) to view the various environmental conditions for your pet (e.g. Labrador Retriever / Dachshund / Yorkshire Terrier / German Shepherd Dog / Golden Retriever) ) / Beagle / Boxer / Poodle / Shih Tzu / Bulldog, Male / Female, Age (1-40), Sleep / Travel You can enter (Travel) / Play (Happy) / Happy / Gentle / Nervous / Fierce).

The controller 250 controls operations of the communicator 110, the feature vector extractor 120, the emotional state analyzer 130, and the user interface 240. To this end, the controller 250 may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). . The controller 160 may execute, for example, an operation or data processing related to control and / or communication of at least one component.

The storage unit 260 stores data for operations of the feature vector extractor 210, the emotional state analyzer 230, and the user interface 240. To this end, the storage unit 260 stores a representative emotion classification value, a sound frequency vector, a sound interval time vector, and the like.

8 is a flowchart of an embodiment for explaining a method for providing information on an emotional state of a companion animal according to the present invention.

First, the client collects the voice information of the companion animal and transmits it to the interpretation server (step S300).

The client continuously detects the pet's voice information, i.e., barking sound or speech. The client sorts the continuously detected sounds in chronological order, continuously detects the sounds in a vector form up to an nth order, and transmits the detected voice information to an interpreter server. The client converts the collected analog signal into a digital signal to detect the voice information of the pet's bark, and then detects the sound information, which is the time interval between the bark and the sound interval, from the converted digital signal.

After the step S300, the interpreter server extracts the voice feature vector from the voice information transmitted from the client (step S302). The interpretation server may extract the speech feature vector using any one of an auto-encoder method and a Mel Frequency Cepstral Coefficient (MFCC) method. The auto-encoder method is an algorithm that derives important features of the data in the process of outputting the same result as the input value. In addition, the MFCC (Mel Frequency Cepstral Coefficient) method is a technique of extracting a feature of a sound, and dividing it into a predetermined time (Short time) of the input sound, and is a technique of extracting a feature by analyzing the spectrum of the section.

After the step S302, the interpretation server performs machine learning using the extracted voice feature vector as input data to extract the emotional state information corresponding to the companion animal (step S304).

The interpretation server may extract the emotional state information using any one of a machine learning algorithm of a deep neural network (DNN) algorithm and a recurrent neural network (RNN) algorithm.

The Recurrent Neural Network (RNN) algorithm receives continuous data such as voice and outputs the result value considering the previous input value. The deep neural network (DNN) algorithm corresponds to a deep neural network, which means an artificial neural network (ANN) including multiple hidden layers between an input layer and an output layer. do.

The interpretation server may infer the emotion by comparing the detected speech feature vector and the pattern of the emotion-specific speech feature vector. For this purpose, the interpreter server determines the most similar emotions by using two or more pattern comparison algorithms such as Dynamic Time Warping (DTW), Hidden Marcov Model (HMM), Artificial Neural Network (ANN), and Fuzzy. Recognize emotional state information about.

After the step S304, the interpreter server transmits the extracted emotional state information to the client (S306). The interpretation server may transmit the emotional state information to the client by wire or wireless communication.

After step S306, the client outputs the emotional state information transmitted from the interpretation server (step S308). The client may output the emotion state information as either a voice signal or an image signal. The client outputs the pet's emotional behavior, dialogue sentence, and dialogue voice. The client can continuously play the 2D image file form to produce a 3D motion effect or drive a 3D motion animation file to generate the motion.

Apparatus and method for analyzing the emotional state of a companion animal according to the present invention described an example applied to the iPhone / iPod touch to download and implement the program through a detailed description, but various portable communication terminals such as a dedicated terminal or a smart phone installed with the program It will be apparent to those skilled in the art that the present invention can be applied to various game machines or the like which are widely used in the related art.

As described above, a preferred embodiment according to the present invention has been described, and the fact that the present invention can be embodied in other specific forms in addition to the above-described embodiments without departing from the spirit or scope thereof is known to those skilled in the art. It is obvious to those who have it.

Therefore, the above-described embodiments should be regarded as illustrative rather than restrictive, and thus, the present invention is not limited to the above description but may be modified within the scope of the appended claims and their equivalents.

10: System for providing information about the emotional state of pets
100: client
200: interpreter server
210: communication unit
220: feature vector extraction unit
230: emotional state analysis unit
240: user interface unit
250: control unit
260: storage unit

Claims (8)

Client for collecting and transmitting the voice information of the companion animal; And
Receiving the collected voice information from the client, and based on the received voice information, the interpreting server for analyzing the emotional state information of the companion animal and transmits to the client,
The interpreter server,
A communication unit for receiving the voice information transmitted from the client;
A feature vector extracting unit extracting a voice feature vector from the received voice information;
An emotional state analyzing unit configured to extract the emotional state information corresponding to the companion animal by performing a machine learning using the voice feature vector extracted by the feature vector extracting unit as input data;
A controller for controlling operations of the communication unit, the feature vector extracting unit, the emotional state analyzing unit, and the state information output unit; And
A storage unit for storing data for the operation of the feature vector extracting unit, the emotional state analyzing unit, the state information output unit, and the control unit;
And the communication unit transmits the emotion state information analyzed by the emotion state analysis unit to the client under the control of the controller. The method according to claim 1,
The feature vector extraction unit,
An information providing system for an emotional state of a companion animal, comprising extracting the speech feature vector using any one of an auto-encoder and a Mel Frequency Cepstral Coefficient (MFCC). The method according to claim 1,
The emotional state analysis unit,
An information providing system for an emotional state of a companion animal, comprising extracting the emotional state information using any one of a machine learning algorithm of a deep neural network (DNN) algorithm and a recurrent neural network (RNN) algorithm. The method according to claim 1,
The communication unit,
And the emotional state information is transmitted to the client by wire or wireless communication. Collecting, by the client, voice information of the companion animal and transmitting it to an interpretation server;
Extracting, by the interpreter server, a speech feature vector from the speech information transmitted from the client;
Extracting, by the interpreter server, emotional state information corresponding to the companion animal by performing machine learning using the extracted voice feature vector as input data;
Transmitting, by the interpreter server, the extracted emotional state information to the client; And
And providing, by the client, the received emotional state information to the emotional state of the companion animal. The method according to claim 5,
Extracting the speech feature vector,
Method for providing information on the emotional state of the companion animal, characterized in that for extracting the speech feature vector using any one of an auto-encoder and an Mel Frequency Cepstral Coefficient (MFCC). The method according to claim 5,
Extracting the emotional state information,
A method for providing information on an emotional state of a companion animal, comprising extracting the emotional state information using a machine learning algorithm of any one of a deep neural network (DNN) algorithm and a recurrent neural network (RNN) algorithm. The method according to claim 5,
Transmitting the emotional state information,
And providing the emotional state information to the client using wired or wireless communication.

Images