KR20200055467A - Artificial intelligence based necklace type wearable device - Google Patents

Abstract

The present invention relates to an artificial intelligence based necklace type wearable device. The artificial intelligence based necklace type wearable device according to an embodiment of the present invention comprises: a communication part for connection with a terminal or server; at least one microphone acquiring sound generated from a wearer; a sound recognition module outputting a recognition result for each of at least one partial sound included in the sound; and a controller that processes the sound acquired from each of the at least one microphone, provides the processed sound to the sound recognition module, and generates information related to a set operation mode based on the recognition result output from the sound recognition module. Therefore, a wearer can conveniently acquire various useful information only by wearing the wearable device.

Description

ARTIFICIAL INTELLIGENCE BASED NECKLACE TYPE WEARABLE DEVICE

The present invention relates to a wearable device, and more particularly, to an artificial intelligence-based necklace type wearable device that recognizes various sounds generated from the wearer's neck.

As society becomes more interested in quality of life, people are paying attention and effort to their health care. In line with this trend, various devices have recently emerged that enable people to check their health status without visiting a medical institution.

In particular, wearable devices that are worn on a part of a person's body to sense and provide information related to the wearer's health state have been released, or concepts related thereto have emerged.

In this regard, in the prior art 1 (Registration Patent Publication No. 10-1607548, registered on March 24, 2016), information related to eating habits using the time required for eating, the number of round trips of the hand or the movement trajectory using the position sensor and the acceleration sensor A wristwatch type wearable communication terminal that provides a device is disclosed. However, in the case of the prior art 1, since information related to the eating habits is provided through indirect information such as hand movement, it may be difficult to provide accurate information, and items of available information may be limited.

Meanwhile, in recent years, interest in artificial intelligence technology has increased. Machine learning exists as a technology for realizing a human learning ability on a computer among various fields of the artificial intelligence technology.

Conventional machine learning was centered on statistical-based classification, regression, and clustering models. In particular, in supervised learning of classification and regression models, a learning model that distinguishes the characteristics of the training data and new data based on these characteristics is previously defined. On the other hand, deep learning, in which interest is increasing along with the recent emergence of the concept of big data, is a computer that finds and discriminates characteristics by using a large amount of data.

Recently, deep learning frameworks provided as open source related to such deep learning have emerged. Accordingly, for effective learning and recognition, in addition to deep learning algorithms, learning processes, learning methods, and extraction and selection of data used for learning are related. Technology is becoming more important. In addition, research to use machine learning for various products or services is increasing.

1. Registered Patent Publication No. 10-1607548 (Registration on March 24, 2016)

The problem to be solved by the present invention is to provide a wearable device capable of more accurately recognizing sound generated from the wearer's neck or mouth, and providing useful information related to the wearer's eating habits and health.

The AI-based necklace type wearable device according to an embodiment of the present invention includes a communication unit for connecting to a terminal or a server, at least one microphone for acquiring sound generated from a wearer, and at least one partial sound included in the sound. Based on the recognition result output from the acoustic recognition module, and the acoustic recognition module for outputting the recognition result, and processing the sound obtained from each of the at least one microphone, providing the processed sound to the acoustic recognition module, And a controller that generates information related to the set operation mode.

The acoustic recognition module includes an acoustic recognizer having a recognition model that recognizes each of the at least one partial sound based on the frequency characteristics of the sound, and the recognition result includes identification information and characteristics of each of the at least one partial sound. Information may be included.

The acoustic recognizer may update the recognition model using frequency characteristics of each of the at least one partial sound and recognition results of each of the at least one partial sound.

According to an embodiment, when the set operation mode is a meal mode, the controller recognizes a meal start time and a meal completion time of the wearer based on the sound acquired from the at least one microphone, and the meal start time and The sound acquired between the meal completion times may be processed, and the processed sound may be provided to the sound recognition module.

The controller controls the communication unit to generate eating habit information for the wearer and transmit the generated eating habit information to the terminal or the server based on a recognition result for each of the at least one partial sound included in the sound. can do.

The eating habits information includes information on at least one item related to the wearer's eating state or behavior during the meal, wherein the at least one item includes a meal time, a conversation time during a meal, a number and intake of water during a meal, and food. It may include at least one of the number of times to chew and the time required to swallow.

The necklace type wearable device may further include an audio output unit that outputs a notification based on the generated eating habits information.

According to an embodiment, when the set operation mode is a non-meal mode, the controller recognizes a state in which the wearer is not eating, based on the sound acquired from the at least one microphone, and for a predetermined time according to the recognition result The acquired sound may be processed, and the processed sound may be provided to the sound recognition module.

The controller generates the respiratory management information for the wearer based on the recognition result for each of the at least one partial sound included in the sound, and transmits the generated respiratory management information to the terminal or the server. Can be controlled.

The respiratory management information includes information on at least one item related to the wearer's respiratory status or moisture ingestion, and the at least one item includes at least one of the number, frequency, or time of occurrence of cough or sputum, and the amount of water intake. It may include.

According to an embodiment, when the set operation mode is a feeding monitoring mode, the controller recognizes a starting point and a completion point of feeding based on the sound acquired from the at least one microphone, and the starting point and completion point of the feeding The sound acquired between viewpoints may be processed, and the processed sound may be provided to the sound recognition module.

The controller generates communication information for the wearer based on a recognition result for each of the at least one partial sound included in the sound, and transmits the generated feeding monitoring information to the terminal or the server. Can be controlled.

The lactation monitoring information includes information on at least one item related to the wearer's lactation state, and the at least one item may include at least one of a breast milk intake amount, a swallowing number, a feeding time, a slack number, and a cycle. have.

According to an embodiment of the present invention, the necklace type wearable device acquires sound generated from a wearer in various situations and accurately identifies various partial sounds related to the situation from the sound through the AI-based sound recognition module 170 can do. The wearable device may generate and provide information based on the identified partial sound and characteristics of the partial sound. That is, the wearer can conveniently obtain various useful information only by wearing a wearable device.

1 is a conceptual diagram of a system including a wearable device and a terminal and a server connected thereto according to an embodiment of the present invention.
2 is a schematic block diagram of a wearable device according to an embodiment of the present invention.
3 is an exemplary view of a necklace type wearable device according to an embodiment of the present invention.
4 and 5 are diagrams for explaining deep learning as an example of artificial intelligence applied to an acoustic recognition module of a wearable device according to an embodiment of the present invention.
6 is a view for explaining a recognition operation and a learning operation of the acoustic recognition module.
7 is a flowchart illustrating an operation of a wearable device according to an embodiment of the present invention.
FIG. 8 is an exemplary view illustrating an embodiment in which the wearable device provides eating habit information from recognized sound in relation to the operation of the wearable device shown in FIG. 7.
FIG. 9 is an exemplary view illustrating an embodiment in which the wearable device provides respiratory management information from recognized sound in relation to the operation of the wearable device shown in FIG. 7.
FIG. 10 is an exemplary diagram illustrating an embodiment in which the wearable device provides feeding monitoring information from recognized sound in relation to the operation of the wearable device shown in FIG. 7.
FIG. 11 is an exemplary diagram illustrating another embodiment in which the wearable device provides feeding monitoring information from the recognized sound in relation to the operation of the wearable device shown in FIG. 7.

Hereinafter, exemplary embodiments disclosed herein will be described in detail with reference to the accompanying drawings. The accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed in the specification is not limited by the accompanying drawings, and all changes and equivalents included in the spirit and technical scope of the present invention It should be understood to include water to substitutes.

1 is a conceptual diagram of a system including a wearable device and a terminal and a server connected thereto according to an embodiment of the present invention.

Referring to FIG. 1, the wearable device 10 may be implemented as an electronic device that can be worn on a specific part of the user's body.

For example, the wearable device 10 may be a necklace type wearable device worn on a user's neck. The necklace type wearable device may include a wireless headset that outputs sound.

Meanwhile, since the wearable device 10 is worn on the user's neck, the wearable device 10 may be in contact with the user's neck or positioned adjacent to the neck when worn.

Accordingly, the wearable device 10 can effectively acquire sound (sound) generated from the user's neck (airway or esophagus) or mouth. The wearable device 10 may use the acquired sound to provide information such as a wearer's eating habits and respiratory condition. In addition, the wearable device 10 may be worn on an infant and provide parents with monitoring information of a lactation state when breastfeeding.

In particular, the wearable device 10 may acquire the recognition result including the identification information or the characteristic information of the sound by recognizing the acquired sound using an artificial intelligence-based recognition module. The wearable device 10 may provide the various information based on the recognition result. For example, the recognition module may include an artificial neural network trained by deep learning.

The configuration and operation of the wearable device 10 will be described in more detail through FIGS. 2 to 11 later.

The terminal 20 may be connected to the wearable device 10 to transmit and receive various data to and from the wearable device 10. The terminal 20 may mean a mobile terminal such as a smart phone or a tablet PC, but may include a fixed terminal such as a PC depending on the embodiment.

In particular, the wearable device 10 may transmit various information based on the recognition result of the sound to the terminal 20. The terminal 20 may provide the information to the user by outputting information received from the wearable device 10 through an output means such as a display or a speaker.

The server 30 may be connected to the wearable device 10 or the terminal 20 to receive various information based on the recognition result of the sound from the wearable device 10. The server 30 may include a database server that stores the received information. According to an embodiment, the server 30 may include a control server that automatically controls the operation of the user's home appliance based on the received information. Alternatively, the server 30 may include a server of a medical service provider that monitors or manages a user's health condition based on the received information. Alternatively, the server 30 learns various sounds obtained from the wearable device 10 and provides learning data generated according to the learning result to the wearable device 10 to update the recognition module of the wearable device 10 It may also include a learning server.

2 is a schematic block diagram of a wearable device according to an embodiment of the present invention, and FIG. 3 is an exemplary view of a necklace type wearable device according to an embodiment of the present invention.

Referring to FIG. 2, the wearable device 10 may include a communication unit 110, an input unit 120, a sensor unit 130, an output unit 140, a memory 150, and a controller 160. The components shown in FIG. 2 are not essential in implementing the wearable device 10, and the wearable device 10 may include more or less components than those shown in FIG. 2.

The communication unit 110 may include at least one communication module that connects the wearable device 10 to the terminal 20 or the server 30. For example, the communication unit 110 may include a wireless Internet module such as Wi-Fi or a short-range wireless communication module such as Bluetooth.

The input unit 120 may include at least one input means for inputting control commands or information related to the operation of the wearable device 10 to the wearable device 10 based on an action such as a user's manipulation. For example, the input means may include a button, a touch pad, a dial, and the like.

In particular, the user may perform an input operation for setting an operation mode to be described later in FIG. 7 through the input unit 120. For example, the operation mode may correspond to any one of a meal mode, a non-meal mode, and a lactation monitoring mode, but is not limited thereto. The wearable device 10 may perform an operation for recognizing sound and an operation for generating information based on the recognition result, based on the set operation mode.

The sensor unit 130 may include a plurality of microphones 131 to 133 that sense sound (sound) generated from the wearer's neck (airway or esophagus) or mouth of the wearable device 10. The control unit 160 may acquire sound from each of the plurality of microphones 131 to 133, and obtain more accurate sound through synthesis and noise removal of the acquired sound. The acquired sound may include various types of partial sounds.

Referring to FIG. 3, the housing 101 of the wearable device 10 may be formed in a wearable form (eg, a necklace form) on the wearer's neck. The plurality of microphones 131 to 133 provided in the housing 101 may be disposed at positions spaced apart from each other. For example, the first microphone 131 and the second microphone 132 may be disposed adjacent to the wearer's vocal cord when worn, and the third microphone 133 may be disposed adjacent to the wearer's neck when worn. That is, the plurality of microphones 131 to 133 are disposed to be adjacent to the wearer's neck, so that sound generated from the wearer's neck or mouth can be accurately obtained. In addition, the plurality of microphones 131 to 133 may acquire the sound at various locations, and the controller 160 may acquire more accurate sound through synthesis of noise obtained at various locations and noise removal.

On the other hand, when the user can input control commands or information in the form of voice through the microphones 131 to 133, the microphones 131 to 133 may function as the input unit 120.

Although not shown, the sensor unit 130 may further include additional sensors (eg, proximity sensors, etc.) other than the microphones 131 to 133.

The output unit 140 may include output means for outputting information or data related to a function provided by the wearable device 10, or outputting information about the state of the wearable device 10.

For example, when the wearable device 10 is implemented as a wireless headset to provide a sound output function, the output unit 140 may include an audio output unit 142 for outputting sound. In addition, the output unit 140 may further include an optical output unit 144 that displays a power state, an operation state, and the like of the wearable device 10.

The memory 150 may store control data or algorithms for the operation of the components included in the wearable device 10. Also, the memory 150 may store sound data received from the terminal 20 or the server 30 to be output through the sound output unit 142.

The controller 160 may control the overall operation of the wearable device 10. The controller 160 may control at least one configuration included in the wearable device 10 to provide a function supported by the wearable device 10 to the user.

The controller 160 may include at least one CPU, application processor (AP), controller, microcomputer, integrated circuit, and the like.

Meanwhile, the controller 160 according to an embodiment of the present invention may recognize the sound by inputting the sound obtained from the plurality of microphones 131 to 133 to the artificial intelligence-based sound recognition module 170. The acoustic recognition module 170 may output identification information of each of the at least one partial sound included in the sound or characteristic information of each of the at least one partial sound as a result of the recognition of the sound.

The identification information is information for identifying the type of sound generated from the user's throat or mouth, and may correspond to, for example, food chewing sound, food swallowing sound, conversation sound, moisture intake sound, cough sound, and the like.

In addition, the characteristic information is a characteristic for each of the identified partial sounds, and may include a generation time, duration, number of times, interval, etc. of the partial sounds.

For example, the acoustic recognition module 170 includes an acoustic recognizer having an artificial neural network (ANN) already learned by machine learning, and includes the identification information or characteristic information by recognizing the input sound. Can output the recognition result.

The controller 160 generates various information based on the recognition result of the acoustic recognition module 170, and outputs the generated information through the output unit 140, or through the communication unit 110, the terminal 20 or the server 30 ).

Hereinafter, deep learning will be described in more detail as an example of artificial intelligence technology applied to the acoustic recognition module 170 with reference to FIGS. 4 to 6.

4 and 5 are diagrams for explaining deep learning as an example of artificial intelligence applied to an acoustic recognition module of a wearable device according to an embodiment of the present invention.

Artificial intelligence is a field of computer science and information technology that studies how computers can do thinking, learning, and self-development that human intelligence can do. Machine learning, one of the fields of artificial intelligence research, may mean a system that performs prediction based on empirical data and improves one's own performance through learning. Deep learning technology, which is a kind of machine learning, is to learn down to a deep level in multiple levels based on data.

Deep learning may represent a set of machine learning algorithms that extract core data from a plurality of data as the level increases.

The deep learning structure may include an artificial neural network (ANN), for example, the deep learning structure may include a deep neural network such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a deep belief network (DBN). (deep neural network).

Referring to FIG. 4, the artificial neural network may include an input layer, a hidden layer, and an output layer. Each layer includes a plurality of nodes, and each layer is connected to the next layer. Nodes between adjacent layers may be connected to each other with a weight.

Referring to FIG. 5, the computing device (machine) may form a feature map by discovering a certain pattern from the input data 510. For example, the computing device (machine) may extract the low-level feature 520, the middle-level feature 530, and the high-level feature 540 to recognize the object and output the result 550.

The artificial neural network can be abstracted to higher level features as it goes to the next layer.

4 and 5, each node may operate based on an activation model, and an output value corresponding to an input value may be determined according to the activation model.

Any node, for example, the output value of the low-level feature 520 may be input to the next layer connected to the node, for example, the node of the middle-level feature 530. The node of the next layer, for example, the node of the middle level feature 530 may receive values output from a plurality of nodes of the low level feature 520.

At this time, the input value of each node may be a value in which weight is applied to the output value of the node of the previous layer. The weight may mean the strength of connection between nodes.

Also, the deep learning process can be regarded as a process of finding an appropriate weight.

Meanwhile, an output value of an arbitrary node, for example, the intermediate level feature 530 may be input to a next layer connected to the corresponding node, for example, a node of the higher level feature 540. The node of the next layer, for example, a node of the higher level feature 540 may receive values output from a plurality of nodes of the intermediate level feature 530.

The artificial neural network may extract feature information corresponding to each level by using a learned layer corresponding to each level. The artificial neural network can be abstracted sequentially, and a predetermined object can be recognized using the highest level of feature information.

For example, looking at the face recognition process by deep learning, the computer distinguishes light and dark pixels from the input image according to the brightness of the pixels, distinguishes simple shapes such as borders and edges, and then uses a more complex shape. You can distinguish things. Finally, the computer can figure out the form that defines the human face.

The deep learning structure according to the present invention can use various well-known structures. For example, the deep learning structure according to the present invention may be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or a Deep Belief Network (DBN).

RNN (Recurrent Neural Network), which is widely used for natural language processing, is an effective structure for processing time-series data that changes with the passage of time, and can build an artificial neural network structure by stacking layers at every moment. .

DBN (Deep Belief Network) is a deep learning structure consisting of a stacked layer of deep learning technique RBM (Restricted Boltzman Machine). By repeating RBM (Restricted Boltzman Machine) learning, if a certain number of layers can be formed, a DBN (Deep Belief Network) having a corresponding number of layers can be constructed.

CNN (Convolutional Neural Network) is a model that simulates a person's brain function based on the assumption that when a person recognizes an object, it extracts the basic characteristics of the object and then performs a complex calculation in the brain to recognize the object based on the result. to be.

On the other hand, the learning of the artificial neural network may be achieved by adjusting the weight of the connection line between nodes (and adjusting the bias value, if necessary) so that a desired output comes out for a given input. In addition, the artificial neural network may continuously update the weight value by learning. In addition, a method such as back propagation may be used for learning the artificial neural network.

Meanwhile, data for learning the artificial neural network may be stored in the memory 150 or the server 30. Further, according to an embodiment, the memory 150 may store weights and biases constituting an artificial neural network structure. Alternatively, according to an embodiment, weights and biases constituting the artificial neural network structure may be stored in the embedded memory of the acoustic recognition module 170.

Meanwhile, whenever the sound acquired from the plurality of microphones 131 to 133 is recognized, the sound recognition module 170 may perform a learning process of the sound recognizer using the acquired sound and the recognition result. As the learning process is performed, the artificial neural network structure such as weight may be updated. To this end, the wearable device 10 may include a learning module. The learning module may be implemented in the acoustic recognition module 170, or may be implemented separately.

6 is a view for explaining a recognition operation and a learning operation of the acoustic recognition module.

Referring to FIG. 6, when a sound is input, the sound recognizer of the sound recognition module 170 may recognize the input sound and output a recognition result. According to an embodiment, the sound recognizer may output the recognition result once as the final recognition result, but repeats or continuously performs multiple recognition operations, and outputs the final recognition result based on the multiple recognition results, thereby inputting the sound It is also possible to improve the recognition accuracy of.

Meanwhile, the acoustic recognizer may collect the received sounds, build a database, and perform learning of the acoustic recognizers using feature points extracted from each of the partial sounds included in the obtained sounds and recognition results of each of the partial sounds. .

For example, the sound recognizer may extract frequency characteristics of each partial sound included in the collected sound. The acoustic recognizer may extract frequency characteristics of each of the partial sounds using a known algorithm such as MFCC (mel-frequency cepstral coefficient). The acoustic recognizer may perform learning and recognition operations of the acoustic recognizer based on the frequency characteristics of each of the partial sounds.

Specifically, the sound recognizer may extract the frequency characteristics of each of the sections by dividing the input sound by a predetermined section and analyzing the frequency spectrum of each of the divided sections. The sound recognizer may recognize the sound of each section by inputting the frequency characteristic (frequency feature vector) of each section to the artificial neural network. The sound recognizer may classify partial sounds included in the acquired sound based on the recognition result, and output a recognition result of each of the partial sounds. Each of the partial sounds may include at least one section.

The acoustic recognizer may perform learning of the acoustic recognizer based on the frequency characteristics of each of the partial sounds included in the acquired sound and the recognition result of each of the partial sounds.

Since the acoustic recognizer recognizes sounds having a frequency pattern that changes over time, the artificial neural network may be composed of a recurrent neural network (RNN), a convolutional neural network (CNN), or a combination thereof.

Through learning the acoustic recognizer, parameters (weight and bias) of the artificial neural network included in the acoustic recognizer may be updated. The database may be stored in the memory 150 of the wearable device 10 or the embedded memory of the acoustic recognition module 170, or may be stored in the memory of the server 30.

Hereinafter, an operation in which the wearable device 10 provides various information through artificial intelligence-based sound recognition according to an embodiment of the present invention will be described with reference to FIGS. 7 to 11.

7 is a flowchart illustrating an operation of a wearable device according to an embodiment of the present invention.

Referring to FIG. 7, the wearable device 10 may set an operation mode related to information to be provided to the user (S100).

In relation to an embodiment of the present invention, the wearable device 10 may include a plurality of modes, and provide different information according to the modes. As described above in FIG. 2, the modes may include a meal mode, a non-meal mode, a feeding monitoring mode, and the like.

The controller 160 receives an input for selecting any one of the plurality of modes through the input unit 120 or the terminal 20, and sets the operation mode of the wearable device 10 to the selected mode. have.

The wearable device 10 may acquire sound through a plurality of microphones 131 to 133 (S110).

The controller 160 may acquire sound from each of the plurality of microphones 131 to 133. Since each of the plurality of microphones 131 to 133 has a different arrangement position, the signal characteristics of the sound acquired by each of the plurality of microphones 131 to 133 may be different.

The controller 160 may perform processing operations such as synthesis and noise removal for each acquired sound. By this processing operation, it is possible to obtain more accurate sound.

Meanwhile, the sound includes at least one partial sound generated from the user during a time during which the user's action related to the set operation mode is performed.

The wearable device 10 may recognize the acquired sound through the artificial intelligence-based sound recognition module 170 (S120).

The controller 160 may input the acquired sound (or processed sound) to the sound recognition module 170. As described above, the acoustic recognition module 170 may include an acoustic recognizer to which an artificial neural network based on machine learning, which is one field of artificial intelligence, is applied.

The acoustic recognition module 170 may recognize identification information and characteristic information of each of the at least one partial sound included in the input sound, and output a recognition result including the recognized information through the sound recognizer. .

The identification information and the characteristic information may be different depending on the operation mode set in step S100. In this case, the acoustic recognizer may include an artificial neural network structure constructed for each of a plurality of modes, but may include an artificial neural network structure integrated for a plurality of modes according to an embodiment.

The wearable device 10 may generate information related to the operation mode set in step S100 based on the recognized sound (S130). The wearable device 10 may transmit the generated information to the terminal 20 or the server 30 (S140).

The controller 160 may generate information related to the operation mode set in step S100, from identification information and characteristic information of each of the at least one partial sound included in the recognition result. The controller 160 may transmit the generated information to the terminal 20 or the server 30 through the communication unit 110.

According to an embodiment, the controller 160 may output the generated information through the output unit 140.

That is, according to an embodiment of the present invention, the wearable device 10 acquires sound generated from a user in various situations, and various partial sounds related to the situation from the sound through the AI-based sound recognition module 170 Can be accurately identified. The wearable device 10 may generate and provide information based on the identified partial sound and characteristics of the partial sound. That is, the user can conveniently obtain various useful information only by wearing the wearable device 10.

Hereinafter, exemplary embodiments in which the wearable device 10 recognizes sound according to various operation modes and provides information based on the recognized sound will be described with reference to FIGS. 8 to 11.

FIG. 8 is an exemplary view illustrating an embodiment in which the wearable device provides eating habit information from recognized sound in relation to the operation of the wearable device shown in FIG. 7.

The embodiment of FIG. 8 corresponds to a case in which the operation mode set in step S100 of FIG. 7 is a meal mode.

Referring to FIG. 8, the wearable device 10 may acquire sound 800 generated from a user's neck or mouth (S800).

As described above in step S110 of FIG. 7, the controller 160 may acquire sound from each of the plurality of microphones 131 to 133 and process the acquired sound to obtain the sound 800.

Meanwhile, the sound 800 may be continuously acquired from a user's meal start time to a completion time. In this regard, the controller 160 may receive a meal start input and a completion input through the input unit 120 to recognize the meal start time and completion time. Alternatively, the controller 160 may recognize the start and finish times of the meal from the characteristics of the sound received from the plurality of microphones 131 to 133, and may recognize the start and finish times of the meal in various other ways. have.

The wearable device 10 may perform the recognition operation of the acquired sound 800 through the artificial intelligence-based sound recognition module 170 (S810).

For example, when the set mode is a meal mode, the acoustic recognition module 170 may recognize the characteristics of partial sounds and partial sounds related to meals from the sound 800.

As illustrated in FIG. 8, the acoustic recognition module 170 may include identification information of each of the at least one partial sound included in the sound 800 (eg, “a sound for chewing hard food”, “a sound for swallowing food”, “ Dialogue sound ”, and“ water swallowing sound ”), and the recognition result 810 including characteristic information of the identified partial sound (eg, frequency or duration of each sound).

The wearable device 10 may generate the user's eating habits information 820 based on the recognition result 810 of the acoustic recognition module 170 (S820).

The eating habits information 820 may include information on a plurality of items related to a meal mode, such as a user's meal state or behavior during a meal. For example, the plurality of items includes the number of chews per minute of food per minute, the number and time of chewing until swallowed by the type of food (hard food / soft food), conversation time during meals, number and intake of moisture during meals, and total meal time. It can, but is not limited to this.

In order to generate information on the plurality of items from the recognition result 810, an algorithm for generating information on each of the plurality of items may be stored in the memory 150.

The wearable device 10 may provide the generated dietary information 820 to the user (S830).

For example, the controller 160 may generate a notification 830 based on the generated dietary information 820 and output the generated notification 830 through the sound output unit 142 or the like.

Alternatively, the controller 160 may transmit the generated dietary information 820 or the notification 830 to the terminal 20 or the server 30. The terminal 20 may output the notification 830 or the received notification 830 based on the received dietary information 820 through a display or a speaker. The server 30 may store the received dietary information 820 or provide it to a medical service server.

That is, according to the embodiment shown in FIG. 8, the wearable device 10 acquires sound generated during the user's meal, and the user's meal state or meal from the sound through the AI-based sound recognition module 170 Various partial sounds related to habits and the like can be accurately identified. The wearable device 10 may generate eating habit information based on the identified partial sound and characteristic information of the partial sound, and provide the generated eating habit information to the user. That is, the user can conveniently obtain accurate eating habits information only by wearing the wearable device 10 during a meal.

FIG. 9 is an exemplary view illustrating an embodiment in which the wearable device provides respiratory management information from recognized sound in relation to the operation of the wearable device shown in FIG. 7.

The embodiment of FIG. 9 may correspond to a case in which the operation mode set in step S100 of FIG. 7 is a non-meal mode.

Referring to FIG. 9, the wearable device 10 may acquire sound 900 generated from a user's neck or mouth (S900).

As described above in step S110 of FIG. 7, the controller 160 may acquire sound from each of the plurality of microphones 131 to 133 and process the acquired sound to obtain the sound 900.

Meanwhile, the sound 900 may be continuously acquired for a predetermined time after the user's meal is completed, or for a predetermined time during which the user is not eating. The controller 160 recognizes that the user's meal has been completed from the characteristics of the sound received from the plurality of microphones 131 to 133, or recognizes that the user is not eating, thereby reproducing the sound of the plurality of microphones 131 to 133 Can be obtained from

The wearable device 10 may perform the recognition operation of the acquired sound 900 through the artificial intelligence-based sound recognition module 170 (S910).

When the set mode is a non-meal mode, the acoustic recognition module 170 may recognize characteristics of partial sounds and partial sounds related to a user's respiratory condition or water intake from the sound 900.

As illustrated in FIG. 9, the acoustic recognition module 170 identifies information of each of at least one partial sound included in the sound 900 (eg, “cough”, “sputum (sputum)”, etc.), and is identified. A recognition result 910 including characteristic information of partial sounds (eg, frequency or duration of each sound) may be obtained.

The wearable device 10 may generate respiratory management information 920 of the user based on the recognition result 910 of the acoustic recognition module 170 (S920).

The respiratory management information 920 may include information on a plurality of items related to a user's respiratory status or water intake status. For example, the plurality of items may include, but is not limited to, the number of coughs, the number of discharges of sputum, the main time of occurrence of cough or sputum, and the amount of water intake.

In order to generate information on the plurality of items from the recognition result 910, an algorithm for generating information on each of the plurality of items may be stored in the memory 150.

The wearable device 10 may provide the generated respiratory management information 920 to the user (S930).

Step S930 may be similar to step S830 of FIG. 8.

Additionally, the controller 160 may transmit respiratory management information 920 to a control server that controls the home appliance in the user's home. The control server may control the operation of a home appliance that controls an air environment such as an air purifier, an air conditioner, and a humidifier based on the received respiratory management information 920. For example, the control server may control the air purifier to activate the air purifying function of the air purifier in a main generation time zone of a user's cough or sputum, based on the respiratory management information 920.

That is, according to the embodiment illustrated in FIG. 9, the wearable device 10 acquires sound generated in a situation other than a user's meal, and the user's respiratory state from the sound through the AI-based sound recognition module 170 However, it is possible to accurately identify various partial sounds related to water intake. The wearable device 10 may generate respiratory management information based on the identified partial sound and characteristic information of the partial sound, and provide the generated respiratory management information to the user. That is, the user can conveniently obtain information related to the respiratory system or water intake by simply wearing the wearable device 10.

FIG. 10 is an exemplary view illustrating an embodiment in which the wearable device provides feeding monitoring information from recognized sound in relation to the operation of the wearable device shown in FIG. 7.

The embodiment of FIG. 10 may correspond to a case where the operation mode set in step S100 of FIG. 7 is a feeding monitoring mode for an infant.

Referring to FIG. 10, the wearable device 10 may acquire sound 1000 generated from the neck or mouth of a user (infant) (S1000).

As described above in step S110 of FIG. 7, the controller 160 may acquire sound from each of the plurality of microphones 131 to 133 and process the acquired sound to obtain the sound 1000.

Meanwhile, the sound 1000 may be continuously acquired while feeding the infant. For example, the controller 160 may recognize that the feeding is started / completed from the characteristics of the sound received from the plurality of microphones 131 to 133, and acquire the sound from the plurality of microphones 131 to 133.

The wearable device 10 may perform the recognition operation of the acquired sound 1000 through the artificial intelligence-based sound recognition module 170 (S1010).

When the set mode is a breastfeeding monitoring mode, the acoustic recognition module 170 recognizes the characteristics of partial sounds and partial sounds related to breastfeeding for infants, such as the number of times the infant has been swallowed, the frequency of swallowing, and the total feeding time. can do.

As illustrated in FIG. 10, the acoustic recognition module 170 may include identification information (eg, “swallow”) of each of the at least one partial sound included in the sound 1000, and characteristic information of the identified partial sound (eg, , Recognition results 1010 including frequency, duration, period, etc. of each partial sound may be obtained.

The wearable device 10 may generate feeding monitoring information 1020 for an infant based on the recognition result 1010 of the acoustic recognition module 170 (S1020).

The breastfeeding monitoring information 1020 may include information on a plurality of items related to an infant's breast milk intake, breastfeeding frequency, and total breastfeeding time.

In order to generate information on the plurality of items from the recognition result 1010, an algorithm for generating information on each of the plurality of items may be stored in the memory 150.

The wearable device 10 may provide the generated feeding monitoring information 1020 to parents, etc. (S1030).

Step S1030 is similar to step S830 of FIG. 8, and a detailed description thereof will be omitted.

That is, according to the embodiment illustrated in FIG. 10, the wearable device 10 acquires sound generated from an infant during feeding for an infant, and breastfeeds the infant from the sound through the artificial intelligence-based sound recognition module 170 Various partial sounds related to intake status can be accurately identified. The wearable device 10 may generate feeding monitoring information based on the identified partial sound and characteristic information of the partial sound, and provide the generated feeding monitoring information to a parent or the like. That is, the parent can conveniently wear the wearable device 10 to the infant, and conveniently obtain the information related to the feeding of the infant.

FIG. 11 is an exemplary diagram illustrating another embodiment in which the wearable device provides feeding monitoring information from the recognized sound in relation to the operation of the wearable device shown in FIG. 7.

Step S1100 of FIG. 11 may be similar to step S1000 of FIG. 10. However, in the present embodiment, the wearable device 10 may acquire the sound 1100 while multiple feedings are performed.

The wearable device 10 may perform the recognition operation of the acquired sound 1100 through the artificial intelligence-based sound recognition module 170 (S1110).

The acoustic recognition module 170 may obtain a recognition result 1110 that recognizes the partial sound and characteristics of the partial sound related to the child's bristle from the sound 1100.

The wearable device 10 may generate feeding monitoring information 1120 for an infant based on the recognition result 1110 of the acoustic recognition module 170 (S1120).

The lactation monitoring information 1120 may include information on the number or cycle of jams during feeding for an infant.

The wearable device 10 may provide the generated feeding monitoring information 1120 to parents, etc. (S1130).

Step S1130 is similar to step S830 of FIG. 8, and a detailed description thereof will be omitted.

According to an embodiment, the wearable device 10, the terminal 20, or the server 30 may further provide a guide for correcting a feeding posture when feeding an infant based on the feeding monitoring information 1120. . For example, in the case where the number of jams from the feeding monitoring information 1120 is greater than the reference number, the wearable device 10, the terminal 20, or the server 30 may provide a guide for reducing the number of jams.

That is, according to the embodiment illustrated in FIG. 11, the wearable device 10 acquires sound generated from an infant during a predetermined number of feedings, and uses the artificial intelligence-based sound recognition module 170 to generate infant sound from the sound. Partial sound related to the number of jams can be accurately identified. The wearable device 10 may generate feeding monitoring information based on the identified partial sound and characteristic information of the partial sound, and provide the generated feeding monitoring information to a parent or the like. That is, the parent can conveniently wear the wearable device 10 to the infant, and conveniently obtain information related to whether the feeding posture for the infant is correct.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains may make various modifications and variations without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments.

The scope of protection of the present invention should be interpreted by the claims below, and all technical spirits within the equivalent range should be interpreted as being included in the scope of the present invention.

Claims (13)

A communication unit for connecting with a terminal or a server;
At least one microphone that acquires sound generated from the wearer;
An acoustic recognition module that outputs a recognition result for each of the at least one partial sound included in the sound; And
Processing sound acquired from each of the at least one microphone, and providing the processed sound to the sound recognition module,
A necklace type wearable device including a controller that generates information related to a set operation mode based on a recognition result output from the acoustic recognition module. According to claim 1,
The acoustic recognition module,
And an acoustic recognizer having a recognition model that recognizes each of the at least one partial sound based on the frequency characteristics of the sound,
The recognition result includes necklace type wearable device including identification information and characteristic information of each of the at least one partial sound. According to claim 2,
The acoustic recognizer,
A necklace type wearable device that updates the recognition model using frequency characteristics of each of the at least one partial sound and a recognition result of each of the at least one partial sound. According to claim 1,
The set operation mode is a meal mode,
The controller,
Based on the sound obtained from the at least one microphone to recognize the meal start time and the meal completion time of the wearer,
A necklace type wearable device that processes the sound acquired between the meal start time and the meal completion time and provides the processed sound to the sound recognition module. The method of claim 4,
The controller,
Based on the recognition result for each of the at least one partial sound included in the sound, the eating habits information for the wearer is generated,
Necklace type wearable device for controlling the communication unit to transmit the generated eating habits information to the terminal or the server. The method of claim 5,
The eating habits information includes information on at least one item related to the wearer's eating state or eating behaviors,
The at least one item is a necklace type wearable device including at least one of a meal time, a conversation time during a meal, a number and intake amount of water during a meal, and a number of chews and a time required to swallow food. The method of claim 5,
A necklace type wearable device further comprising an audio output unit outputting a notification based on the generated eating habits information. According to claim 1,
The set operation mode is a mode other than meal,
The controller,
Based on the sound obtained from the at least one microphone, the wearer recognizes the state of not eating,
A necklace type wearable device that processes sound acquired for a predetermined time according to the recognition result and provides the processed sound to the sound recognition module. The method of claim 8,
The controller,
Based on the recognition result for each of the at least one partial sound included in the sound, generate respiratory management information for the wearer,
Necklace type wearable device for controlling the communication unit to transmit the generated respiratory management information to the terminal or the server. The method of claim 9,
The respiratory management information includes information on at least one item related to the wearer's respiratory status or moisture ingestion,
The at least one item is a necklace type wearable device that includes at least one of the number, frequency, or occurrence time of coughing or sputum, and water intake. According to claim 1,
The set operation mode is a feeding monitoring mode,
The controller,
Recognizing the start and finish time of feeding based on the sound obtained from the at least one microphone,
A necklace type wearable device that processes the sound acquired between the start time and the completion time of the feeding and provides the processed sound to the sound recognition module. The method of claim 11,
The controller,
Based on the recognition result for each of the at least one partial sound included in the sound, generates feeding monitoring information for the wearer,
Necklace-type wearable device that controls the communication unit to transmit the generated feeding monitoring information to the terminal or the server. The method of claim 12,
The feeding monitoring information includes information on at least one item related to the wearer's feeding status,
The at least one item is a necklace-type wearable device that includes at least one of a breast milk intake amount, a swallowing number, a feeding time, and a number of times of hunger.

Images