KR20180094163A - Method and apparatus for recognizing activities through spatial filtering and machine learning - Google Patents

Abstract

The present invention relates to a method for recognizing an action through spatial filtering and machine learning and an apparatus thereof. The method and apparatus can remarkably lower time required for recognition and computational complexity and derive a recognition result with high reliability by dividing space of a house into a plurality of spaces, filtering data received from the sensors located in the house per the divided space, and recognizing an action of a user using only sensor data filtered in the space while the user is located in a specific space. Specifically, the method and apparatus generate a learning model learning various predefined user actions to recognize the action of the user through machine learning and recognize the actions of the user by inputting the sensor data on the actions of the real use to the generated learning model.

Description

[0001] METHOD AND APPARATUS FOR RECOGNIZING ACTIVITY THROUGH SPATIAL FILTERING AND MACHINE LEARNING [0002]

The present invention relates to a method and apparatus for recognizing an action through spatial filtering and machine learning. More specifically, the present invention relates to a method and apparatus for recognizing an action through spatial filtering and machine learning, It is possible to reduce the time and computational complexity required for the recognition and to improve the reliability of the recognition result And to provide a method and apparatus that can derive the same. In particular, the present invention generates a learning model in which a variety of user behaviors defined in advance are learned through machine learning in order to recognize the behavior of the user, inputs sensor data on an actual user's behavior to the generated learning model And recognizes the behavior of the user.

Recently, due to the development of sensor technology and the development of information communication technology, all information communication technologies are converged to one device or service, and various kinds of communication networks such as wired / wireless network, broadcasting communication network and the like are connected together. Accordingly, the Internet of Things (IoT) technology, which enables various kinds of objects including chips and sensors to be connected to the Internet to produce information and provide it to users, .

Object Internet technology is an extension of the person-centered information utilization method based on the communication between people and people to the area between people and objects or things and objects. It can connect people, devices, space, data, Means an intelligent technology or service that has the function of exchanging data as an organism between objects and objects as well as people and objects, and enables mutual communication at anytime and anywhere.

In other words, the object Internet technology can recognize the situation of the user before the object needs the user, acquire the information, and provide the contents necessary for the user through the analysis.

According to this social change, the activity pattern of the user located in the house is collected by the activity recognition technology utilizing the plural sensor and the object Internet technology to recognize the behavior of the user, Technologies for providing customized services have been developed and commercialized. For example, there is a system that monitors the daily life of the elderly living alone and recognizes the activity of daily living (ADL) of the elderly living alone using the human body sensor.

Meanwhile, the conventional behavior recognition technology recognizes the user's behavior by measuring the position of the user using Wi-Fi (wireless fidelity) technology and RFID (radio frequency identification) technology.

However, the behavior recognition technology using Wi-Fi technology requires specific actions such as having the user always have a smart phone in order to grasp the location of the user, .

In addition, the conventional behavior recognition technology using RFID technology allows a user to wear the RFID reader directly and recognize the user's behavior while reading the tag contents from the article attached with the RFID tag. However, since the behavior recognition technology using the RFID technology requires a user to wear the RFID reader directly, there is a limit to be practically applied. In the case of attaching the RFID tag to an object such as metal, the RFID recognition performance itself is deteriorated.

In order to solve such a problem, an action recognition system has been developed to recognize a user's behavior based on sensor data collected from the sensor using various household appliances equipped with sensors.

The existing behavior recognition system using home appliances equipped with sensors collects sensor data received from all the sensors installed in the house and recognizes the behavior of the user through a collective reasoning process. Therefore, the time and calculation complexity There is a problem that the user's behavior can not be recognized in real time.

In addition, existing behavior recognition systems have a disadvantage in that their accuracy and reliability are very low because they recognize user's actions by utilizing all sensor data collected without considering direct sensor data for recognizing user's actions.

Accordingly, in the present invention, the architecture for the object Internet defined in oneM2M, which is an international standard for the object Internet, is applied to perform spatial filtering for recognizing the behavior of the user based on the container concept of the resource type related to the spatial information and the sensor , Recognizing the behavior of the user through sensor data received from at least one sensor located in a specific space in which the user is perceived and a learning model learning various predefined behaviors of the user, The present invention provides a method and an apparatus for remarkably improving the accuracy and reliability of the recognition result while realizing the user's behavior in real time by drastically reducing the complexity.

Next, a brief description will be given of the prior arts that exist in the technical field of the present invention, and technical matters which the present invention intends to differentiate from the prior arts will be described.

Korean Patent Laid-Open Publication No. 2009-0019152 (Feb. 25, 2009) discloses a method and system for recognizing daily life activities of elderly people by installing sensors at necessary positions in a home. The present invention relates to a method and system for recognizing an action of an elderly person in daily life by converting the received measurement data into a basic action corresponding to a sensor that has transmitted the measurement data.

The prior art is somewhat similar to the technical construction of the present invention in that it recognizes the behavior of the user through a plurality of sensors.

On the other hand, according to the present invention, a house space is divided into at least one space for each feature through spatial filtering, and a behavior of a user is recognized based on sensor data received through a separate message container for a space divided for each feature When a user is sensed in a specific space, only the sensor data measured by the plurality of sensors located in the specific space can be used to recognize the behavior of the user in the space. In other words, by allowing sensor data other than the specific space in which the user is located to be recognized so that the user's behavior can be recognized, it is possible to maintain high accuracy and reliability of the recognition result, .

However, the above-described prior art has previously set the basic behavior of the elderly person for eating, sleeping, resting, and toilet, and when the measurement data is received from the sensor related to the basic action, the behavior of the elderly person is changed into eating, sleeping, Only the configuration to be recognized is described. However, according to the present invention, the home space is divided into a plurality of spaces, the behavior of the user in each space is recognized, and according to the association between the plurality of sensors (i.e., The order of the measured time) does not provide a technical configuration that allows the user to recognize an action including a movement, an action, or a combination thereof, so that the present invention and the prior art can easily identify the different technical configurations .

Korean Patent Registration No. 0913396 (Aug. 14, 2009) relates to an apparatus and method for recognizing an action using a motion vector. The motion recognition apparatus calculates an average motion vector for X and Y axes in an image input through a camera, And recognizing the behavior of the user, and a method thereof. To this end, the prior art stores behavior type information corresponding to an average motion vector, calculates a motion vector for the object after recognizing the object from the image collected from the camera, and outputs the stored motion vector The recognition table is searched to recognize the behavior of the user.

The prior art recognizes the behavior of the user and analyzes the moving picture inputted from the camera to recognize the behavior of the user, Its processing complexity is very high, processing time is long, and there is a limit to recognize correct behavior.

On the other hand, the present invention separates a house space into a plurality of spaces according to a characteristic (for example, a use purpose), filters sensor data received from a plurality of sensors located in the house by each separated space, And recognizes the behavior of the user located in the user. This is because, unlike the prior art, only the sensor data filtered into the space is used in recognizing an action for a user located in a specific space, thereby greatly reducing the time and computational complexity required for the recognition, It is possible to remarkably improve the accuracy.

It is an object of the present invention to provide a method and apparatus for recognizing a user's behavior based on sensor data received from a plurality of sensors installed in a home.

In addition, the present invention provides a method and apparatus for separating a house space into a plurality of spaces according to characteristics and realizing a user's behavior through spatial filtering processing only sensor data of a corresponding space in which the user is located .

In addition, the present invention predefines an action generated by a user for each of the separated spaces and allows the user to recognize the behavior of the user by excluding sensor data not related to the defined action for each space, And to provide a method and apparatus for dramatically reducing time and computational complexity.

The present invention also provides a method and apparatus for detecting a user's behavior by selecting a most probable action among a plurality of predefined actions based on a plurality of sensor data located in the separated space by applying machine learning A method and an apparatus are provided.

A device for recognizing a behavior through spatial filtering and machine learning according to an embodiment of the present invention includes a spatial filtering unit for filtering a plurality of sensor data received from a house separated into a plurality of spaces by the separated spaces, And a user action recognizing unit recognizing an action of a user located in a specific space by using the sensor data separately.

Also, the sensor data is named and received by the names of the separated spaces, and the filtering by the spaces is performed by classifying the sensor data by spaces referring to the names of the spaces that are named in the sensor data .

The behavior recognition apparatus may further include a learning model generation unit configured to learn a behavior recognition data set for recognizing user behavior and generate a learning model for each of the separated spaces, Information about a space, at least one activity generated by a user according to each space, and sensor information located in each space.

The learning model may include a combination of a plurality of sensor data filtered for each space, and calculates and outputs a probability of at least one activity according to each space.

The user behavior recognition unit may be configured to input a combination of sensor data filtered in a corresponding space to a learning model for the specific space to recognize an action of a user located in the specific space, And recognizing an action of a user located in the specific space by selecting an activity having a maximum possibility among the possibilities.

According to another aspect of the present invention, there is provided a method of recognizing an action through spatial filtering and machine learning, the method comprising: a spatial filtering step of filtering a plurality of sensor data received from a house separated into a plurality of spaces by the separated spaces; And recognizing an action of a user located in a specific space by using space-specific sensor data.

The behavior recognition method further includes a learning model generation step of learning a behavior recognition data set for recognizing user behavior and generating a learning model for each of the separated spaces, Information about each space, at least one activity generated by a user according to each space, and sensor information located in each space.

The user action recognizing step may include inputting a combination of sensor data filtered into the corresponding space in the learning model for the specific space to recognize an action of a user located in the specific space, And recognizing an action of a user located in the specific space by selecting an activity having a maximum possibility among the possibilities for the specific space.

As described above, according to the present invention, there is provided a method and apparatus for recognizing an action through spatial filtering and machine learning. The method includes: dividing a house space into at least one space according to a characteristic; By filtering each space, data for recognizing the user's behavior can be minimized, and the time and computational complexity required for the recognition can be drastically reduced.

In addition, the present invention provides a method and apparatus for learning a behavior-aware data set including activities predefined for each space through a machine learning technique for recognizing a user's behavior, generating a learning model for each space, In this case, by recognizing the action of the user by using the sensor data filtered with the specific space and the learning model for the space, movement in the user's home space and action in the user's specific space can be recognized in real time There is an effect that a reliable recognition result can be derived.

1 is a conceptual diagram for schematically explaining a method of recognizing an action through spatial filtering and machine learning according to an embodiment of the present invention and an apparatus therefor.
2 is a block diagram illustrating a configuration of an action recognition device for recognizing an action of a user located in a house according to an embodiment of the present invention.
FIG. 3 is a view for explaining a user behavior recognition data set according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating a resource, which is a type of data stored in a behavior recognizing apparatus according to an embodiment of the present invention.
5 is a flowchart illustrating a process of generating a learning model and a process of recognizing a user's behavior according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a method and apparatus for recognizing an action through spatial filtering and machine learning of the present invention will be described in detail with reference to the accompanying drawings. Like reference symbols in the drawings denote like elements. Furthermore, specific structural and functional descriptions for embodiments of the present invention are presented for the purpose of describing an embodiment of the present invention only, and, unless otherwise defined, all terms used herein, including technical or scientific terms Have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as ideal or overly formal in the sense of the art unless explicitly defined herein .

1 is a conceptual diagram for schematically explaining a method of recognizing an action through spatial filtering and machine learning according to an embodiment of the present invention and an apparatus therefor.

As shown in FIG. 1, the behavior recognition apparatus 100 recognizes a user's behavior based on sensor data received from a plurality of sensors 200 located in a house.

The sensor 200 senses information on the state of the object to be measured or information about the surrounding environment, and transmits information about the object. The object Internet device includes at least one sensor and a passive infrared sensor (PIR) , A PIR sensor), a temperature sensor, a humidity sensor, or an ultrasonic sensor, a wearable sensor attached to a user's body to measure a user's heart rate, body temperature, blood pressure, It is a concept that includes combinations.

On the other hand, the object Internet device is connected to the network and includes at least one sensor 200 and is connected to a home appliance such as a TV, a microwave oven or a lamp for transmitting information on a specific action (on or off of the electronic device) And electronic devices.

In addition, the PIR sensor is a sensor for detecting a movement of a user at a specific position, and detects the presence or absence of a user by detecting a change amount with respect to an infrared wavelength distribution radiated from a human body.

That is, the behavior recognizing apparatus 100 recognizes the user's behavior by using sensor data generated and transmitted by the object Internet device, at least one sensor 200 located in the house, or a combination thereof.

Meanwhile, the home space is partially separated into at least one space according to characteristics such as a use purpose or a use range, and the behavior recognition device 100 detects sensor data received from a plurality of sensors 200 located in the house, When the user is located in a specific space, the user's behavior is recognized using only the sensor data filtered into the space.

That is, the spatial filtering divides the indoor space according to the use purpose or the use range such as a kitchen, a living room, a bathroom, a living room, etc., and classifies the sensor data received from the plurality of sensors 200 according to each space, Thereby making it possible to easily recognize an action for a user in a specific space.

Also, the behavior recognizing apparatus 100 has a predefined user behavior recognizing data set for recognizing a user's behavior in a specific space, and the sensor data received from the plurality of sensors 200 Based on the user behavior recognition data set, it recognizes the behavior for the user in a specific space.

The user behavior recognition data set includes a plurality of separated spatial information, a user activity that can be performed in each separated space, and sensor information located in each space. The behavior recognition data set will be described in detail with reference to FIG.

Also, the behavior recognition apparatus 100 learns the user behavior recognition data set defined in advance by applying classification methods according to a machine learning technique for recognizing the behavior of the user, Create a star-learning model.

In addition, when the user is sensed in a specific space, the behavior recognition apparatus 100 inputs the sensor data filtered in the specific space among sensor data received from the plurality of sensors 200 into a learning model corresponding to the specific space And recognizes the behavior of the user. Meanwhile, the learning model may be generated by learning not only the user behavior recognition data set defined in advance but also the user behavior recognition data set, the data by direct input by the user, or a combination thereof, and the contents of the behavior recognition data set Added, changed, or deleted, it may be updated to reflect this. The machine learning will be described in detail with reference to FIG.

The sensor data generated by the plurality of sensors 200 is encoded in a resource format defined by the oneM2M international standard, and the resources are named and transmitted / received for each space. The resource will be described in detail with reference to FIG.

As described above, although the behavior recognition apparatus 100 is primarily intended to recognize the behavior of a user located in a house through a plurality of sensors 200, the behavior recognition apparatus 100 may include a sensor (not shown) And may provide the user with information about the user's living environment such as the operating state of the electronic device based on the data.

Also, the behavior recognizing apparatus 100 may be implemented in the form of a cloud server connected to a network, or may be implemented in a form specific to each member of the home.

2 is a block diagram illustrating a configuration of an action recognition device for recognizing an action of a user located in a house according to an embodiment of the present invention.

2, the behavior recognition apparatus 100 for recognizing an action for at least one user located in a house separated into a plurality of spaces includes a sensor data collection unit (for collecting sensor data from a plurality of sensors) A user detection unit 120 for detecting presence or absence of a user in a specific space based on the collected sensor data, a spatial filtering unit 130 for filtering the collected sensor data by the separated spaces, A learning model generation unit 140 for generating a learning model for recognizing a behavior, a user behavior recognition unit 150 for recognizing a user's behavior using the generated learning model, A sensor data management unit 160 for managing data and a control unit 170 for controlling the behavior recognition apparatus 100 as a whole.

The sensor data collection unit 110 collects sensor data in real time from a plurality of sensors 200 located in the house.

Meanwhile, the sensor data is encoded and transmitted as a resource defined by the oneM2M international standard for the Internet of Things, and the resource is named by the name of each space in which the plurality of sensors 200 transmitting the sensor data are located and transmitted . Also, the behavior recognition apparatus 100 stores sensor data received from the sensor 200 for each space, and stored sensor data is also stored in a resource form. The resource will be described in detail with reference to FIG.

Each of the above-mentioned spaces has been described with reference to FIG. 1 as an entire space in the house is divided according to spatial characteristics, and a detailed description thereof will be omitted.

Also, the user sensing unit 120 senses a user located in a specific space based on the received sensor data.

The user sensing unit 120 senses whether or not a user exists in a specific space. The sensing unit senses sensor data received from a PIR sensor or a sensor 200 triggered by a user in a specific space for the first time . ≪ / RTI >

That is, the sensors 200 generate sensor data in real time according to the purpose of use and transmit the behavior recognition apparatus 100. The behavior recognition apparatus 100 detects the presence of the user through the PIR sensor When the sensor data is received from the sensor data to be transmitted or the sensor 200 that is triggered first by the user to control a specific device or by sensing the user in a specific space, In which space it is located.

For example, when receiving the sensor data from the PIR sensor located in the bathroom, the user detection unit 120 may detect that the user is located in the toilet, and if the user moves to the kitchen When sensor data for the state change of the microwave oven is received from a sensor for sensing the on / off state of the microwave oven when the microwave oven is first turned on, it is determined that the user is located in the kitchen Can be detected.

If it is detected by the user detection unit 120 that the user is located in a specific space, the spatial filtering unit 130 for filtering the sensor data collected in real time according to the space, And provides it to the recognition unit 150.

Accordingly, in recognizing the user's action based on the sensor data, sensor data other than the specific space in which the user is located can be excluded, and the time and computational complexity required for the recognition can be significantly reduced.

On the other hand, the filtering is performed by referring to the space name which is named in the sensor data, and by selecting the named sensor data by the name of the specific space in which the user is located.

Also, the learning model generation unit 140 learns previously stored behavior recognition data sets, and generates a learning model for recognizing the behavior of the user.

Meanwhile, the learning model is generated for each space, and the generated learning model is stored in the database 300 so that it can be used by the user behavior recognition unit 150.

Also, the learning model generation unit 140 periodically learns the behavior recognition data set, or when the data included in the behavior recognition data set is updated by the user such as addition, deletion, change, etc., the updated behavior recognition data And learning models can be kept up-to-date.

Meanwhile, the input of the generated learning model is composed of combinations of the sensor data, and is sequentially input according to the sensing time of the sensor data. Also, the learning model calculates and outputs a possibility for each space activity defined in advance based on a combination of the input sensor data, and the user behavior recognition unit 150 selects an activity having the maximum possibility , And recognizes the behavior of the user. On the other hand, the likelihood can be expressed by probability.

Also, the learning model generation unit 140 learns the behavior-aware data set using a machine learning technique for recognizing the behavior of the user in a specific space by calculating the possibility of the user activity for each space.

Meanwhile, the learning can be performed through a naive bayes (NB) technique, a hidden markov model (HMM) technique, or a hidden semi-markov model (HSMM) technique. However, the present invention is not limited to the above-described machine learning techniques, and various machine learning techniques for learning the behavior of the user by learning the predefined behavior-aware data sets, . ≪ / RTI > That is, the present invention does not limit the machine learning technique.

On the other hand, the NB technique calculates a probability of each classification of an object to be classified by applying Bayes theorem that assumes independence between characteristics, and classifies the probability into a higher probability. For example, When there is a classification for predefined activities, probability of each activity is calculated according to the combination of sensor data to be inputted, and the activity having high probability is selected so that the user's behavior can be recognized in the specific space.

In addition, HMM and HSMM are machine learning methods that are used mainly for dealing with time-dependent sequential data. Probability inference method assumes that observation at a specific time (t) depends only on r most recent observations. And selects an activity having the highest probability among the activities in the predetermined space in accordance with the combination of sensor data received through the sensor data, thereby allowing the user's behavior to be recognized.

In addition, when the user behavior recognition unit 150 receives a plurality of filtered sensor data through a spatial filtering unit 130, a learning model for recognizing a user's behavior in the corresponding space is stored in the database (300).

Also, the user behavior recognition unit 150 inputs a combination of the plurality of provided sensor data to the loaded learning model, and recognizes an action for the user in the corresponding space.

The combination of the plurality of sensor data means a sequence in which a plurality of sensors 200 are operated according to an action of a user. That is, when the sensor # 1, the sensor # 2, and the sensor # 3 are located in a specific space and the sensor # 3, the sensor # 1, and the sensor # 3, sensor # 1 and sensor # 2 are sequentially inputted to the learning model.

On the other hand, the plurality of sensors 200 located in a specific space are not operated sequentially in a series of operation sequences, but are randomly operated according to the user's actions to transmit sensor data. Accordingly, the combination of the sensor data becomes discrete-time series data, and the learning model inputs the data of the discrete time series to calculate the possibility for the predetermined space-specific activity.

Also, the user behavior recognition unit 150 recognizes the user's behavior by selecting an activity having the maximum possibility among the possibilities for each activity calculated by the learning model.

Also, the user behavior recognition unit 150 transmits the recognized result to a communication terminal of another user (for example, a guardian of the recognition subject) connected to the network so that the user can confirm it in real time.

In addition, the sensor data management unit 160 stores and manages sensor data received in real time in the database 300 in the form of a resource. At this time, the sensor data is filtered by the spatial filtering unit 130 and stored in the resources defined for each space.

FIG. 3 is a view for explaining a user behavior recognition data set according to an embodiment of the present invention.

As shown in FIG. 3, the user behavior recognition data set is data that is learned to recognize an action of a user located in a specific space in the house, and is stored in the behavior recognition apparatus 100.

The user behavior recognition data set is data that divides a house space into at least one space according to a space feature including a usage purpose, a range, a size, or a combination thereof, and defines an activity for each space in advance.

In addition, the user behavior recognition data set includes spatial information for each separated space, activity for each space, and information about at least one sensor 200 located in each space.

The space information represents information obtained by separating a space in the house according to characteristics, and may be, for example, a bedroom, a bathroom, a kitchen, or a front door. It is to be understood that this is only one embodiment and can be more finely divided by the user.

In addition, space-specific activities define user activities that can be performed in each space. For example, in the case of a bedroom, "go to bed", "get dressed" . This can also be variously defined depending on the user.

The information on the sensor 200 indicates the types of the sensors 200 located in the corresponding space. For example, a reed switch sensor that detects whether a door and a cupboard are opened or closed, a pressure mats sensor that measures whether the user is sitting on a couch or lying on a bed, A flood sensor for detecting the use of the toilet, and the like.

As described above, the behavior recognition data set is learning data for recognizing the behavior of the user, and the learning model generated through the learning is a combination of the sensor data generated and transmitted by the sensors 200 for each space And calculates and outputs the possibilities for activities in each space.

FIG. 4 is a diagram illustrating a resource, which is a type of data stored in a behavior recognizing apparatus according to an embodiment of the present invention.

Resources, on the other hand, follow the resource structure defined by the oneM2M standard. However, the sensor data received from each of the sensors 200 is named and stored for each space.

As shown in FIG. 4, the resources stored in the behavior recognition apparatus 100 have a structure connected by a tree structure or a link, and addressing is performed using a unique address (e.g., a URI) Quot; data structure "

Sensor data received from a plurality of sensors 200 is stored as a resource, and the resource is identified by a uniform resource identifier (URI). Also, a resource is defined as a resource type, and the resource type is defined as attributes and child resource types of the resource type.

Therefore, as shown in FIG. 4, the resource type < action recognition device > is a root resource type of a resource stored in the behavior recognition device 100, and includes sensor data and service received from the sensor 100, Information about policies to provide is stored.

The resource type < action recognition device > includes an access control policy resource type, a container resource type, a group resource type, a subscription resource, Type, and the like.

The AE resource type is named by the name of each space that divides a house space into a plurality of spaces, and information on the space-dependent sensor 200 and space-specific sensor data are stored.

For example, when the AE resource type is a bathroom (i.e., <AE: bedroom>), the identifier (ID) of the AE resource type, the access control policy resource type, the subscription resource type, For example, a light sensor, a temperature and humidity sensor, an ultrasonic sensor, etc.). In addition, the container resource type is named by the name of each sensor 200 as the data of the sensors 200 located in the bathroom are stored.

The access control policy resource type is also a resource type that stores information related to access rights. That is, access control is performed on the operation of accessing using the access authority information included in the resource type < behavior recognition apparatus >.

The container resource type included in the resource type < behavior recognition device > stores information generated by the behavior recognition device 100. [

In addition, the group resource type provides a function that allows a plurality of resources to be bundled together, and the subscription resource type notifies that a state of a resource is changed through a notification.

5 is a flowchart illustrating a process of generating a learning model and a process of recognizing a user's behavior according to an embodiment of the present invention.

5, in order to create a learning model for recognizing a behavior of a user, the behavior recognition apparatus 100 loads the behavior recognition data set stored in the database 300 (step &lt; RTI ID = 0.0 &gt; S110).

The behavior recognition data set may include a plurality of spaces such as a kitchen, a living room, a bathroom, a living room, a small room, a front door, a bathroom, or the like, and may define activities that can be performed by the user in the space , And information on the plurality of sensors 200 located in the separated plurality of spaces.

Next, the behavior recognition apparatus 100 performs a learning model learning for each space based on the loaded behavior recognition data set (S120), generates a learning model for each of the separated spaces, and stores the generated learning models in the database 300 (S130).

The learning model learning for each space is performed separately for each of the separated spaces and a discrete time series combination of sensor data generated by at least one sensor 200 including the data set ), The possibility of each activity defined for each space can be calculated and output.

For example, if the space is a kitchen and the predefined activity is "eat food." (Sensor # 2) for detecting the operating state of the microwave oven, and a sensor (not shown) for detecting the opening and closing of doors of the refrigerator (Sensor # 3) and a pressure sensor (sensor # 4) for sensing the sitting of a person on a chair, the sensor # 1, the sensor # 2, the sensor # 3 and the sensor # And outputs the possibility for the activity. At this time, the combination of the sensors (sensor # 1, sensor # 2, sensor # 3, and sensor # 4) is composed of at least one.

As described above, it is preferable that the learning model is generated for each space. However, it is possible to implement the learning model by learning the behavior recognition data set and calculating the possibility for the activity for each space through one learning model Do.

Hereinafter, a process of recognizing a user's behavior in a specific space will be described in detail.

As shown in FIG. 5, in the process of recognizing a user's behavior in a specific space, the behavior recognition apparatus 100 collects sensor data in real time from a plurality of sensors 200 located at home (S210).

The sensor data is transmitted in the resource format, and the type of the resource is transferred by naming the specific space in which the sensor 200 transmitting the sensor data is located.

Further, the sensor data includes sensing information (i.e., a measured value), and may further include an identifier (ID) of the sensor 200 and a sensing time. The sensor data is encoded and transmitted in a resource structure.

The behavior recognition apparatus 100 also receives sensor data from a plurality of sensors 200 and inserts the sensor data into a resource structure managed by itself, stores the sensor data in the database 300, manages the sensor data, 200 may be updated by reflecting a change in the measured value of the received sensor data.

Next, the behavior recognition apparatus 100 senses a user based on the plurality of received sensor data (S220).

The sensing is performed to recognize a specific space in which the user is located. The specific space in which the user is located is a PIR sensor that is located in the specific space and senses the motion of the user, The sensor data transmitted from the sensor 200 can be detected.

Next, the behavior recognition apparatus 100 selects sensor data transmitted from the sensors 200 located in the specific space sensed in the step S220 among the sensor data received from the sensors 200 in the house through spatial filtering (S230).

Meanwhile, the spatial filtering refers to filtering the sensor data received from a plurality of sensors 200 located in the house by the separated spaces, by separating the house space into a plurality of spaces in advance, It is possible to recognize the behavior of the user for each space. This makes it possible to recognize the behavior of the user by using only the sensor data that is substantially required by excluding the sensor data unnecessary for the recognition, thereby remarkably reducing the time and the calculation complexity and deriving a more accurate and reliable recognition result .

Filtering the sensor data for each separated space is performed by referring to the space name that is nimmed in the sensor data and classifying the sensor data according to the referenced space name.

In addition, the sensor data filtered by the space is stored in resources for the separated spaces managed by the behavior recognition apparatus 100, respectively.

Next, the behavior recognition apparatus 100 recognizes the user's action using the sensor data for the specific space filtered in operation S230 (S240).

In addition, the behavior recognition apparatus 100 loads a learning model for the space from the database 300 to recognize an action for a user in a specific space.

Then, the behavior recognition apparatus 100 inputs a combination of the filtered sensor data to the loaded learning model, and recognizes the user's behavior according to the result output by the learning model.

Meanwhile, the combination of the sensor data means a combination according to the sensing time of the sensor data (that is, the operation sequence of the sensors 200 that transmitted the sensor data) The sensor data is sequentially input to the learning model in accordance with the learning model.

In addition, the output of the learning model is a value for the possibility of each activity by a space defined in advance, and the behavior recognition apparatus 100 selects an activity having a value for the maximum possibility according to the output of the learning model, Recognize the behavior of the user.

Next, the behavior recognizing apparatus 100 outputs the recognized result in step S240, and provides the result to the communication terminal provided by the user (S250). At this time, the user may be a guardian or a manager of the subject including the person to be recognized for the action.

As described above, the apparatus and method for recognizing an action through spatial filtering and machine learning of the present invention generate learning models by learning various activities that can be generated by a user in a predetermined space defined in advance, The user's behavior can be recognized in real time by inputting only the sensor data received from the specific space in which the user is located to the learning model, and the time and calculation complexity required for the recognition can be remarkably reduced.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, I will understand the point. Accordingly, the technical scope of the present invention should be determined by the following claims.

100: Action recognition device 110: Sensor data collection unit
120: user sensing unit 130: spatial filtering unit
140: Learning model generation unit 150: User behavior recognition unit
160: sensor data management unit 170:
200: sensor 300: database

Claims (10)

A spatial filtering unit for filtering a plurality of sensor data received from a house separated into a plurality of spaces by the separated spaces; And
And a user behavior recognizing unit that recognizes an action of a user located in a specific space by using the filtered sensor data for each space. The method according to claim 1,
The sensor data includes:
A name of each space is received and received,
Filtering by the space may include:
And classifying the sensor data by spaces with reference to a name of a space that is named in the sensor data. The method according to claim 1,
The behavior recognition apparatus includes:
And a learning model generation unit that learns an action recognition data set for recognizing user behavior and generates a learning model for each of the separated spaces,
Wherein the behavior recognition data set includes information on the separated space, at least one activity generated by a user according to the space, and sensor information located in each space. . The method of claim 3,
The learning model includes:
Wherein a combination of a plurality of sensor data filtered by each space is inputted and the possibility of at least one activity according to each space is calculated and output. The method of claim 4,
The user behavior recognizing unit,
A combination of sensor data filtered in a corresponding space is input to a learning model for the specific space in order to recognize a behavior of a user located in the specific space and a maximum likelihood And recognizing an action of a user located in the specific space by selecting an activity. A spatial filtering step of filtering a plurality of sensor data received from a house separated into a plurality of spaces by the separated spaces; And
And a user action recognition step of recognizing an action of a user located in a specific space by using the filtered sensor data for each space. The method of claim 6,
The sensor data includes:
A name of each space is received and received,
Filtering by the space may include:
And classifying the sensor data by a space by referring to a name of a space that is named in the sensor data. The method of claim 6,
The action recognition method includes:
And a learning model generation step of learning a behavior recognition data set for recognizing user behavior and generating a learning model for each of the separated spaces,
Wherein the behavior recognition data set includes information on the separated space, at least one activity generated by a user according to the space, and sensor information located in each space. A method of recognizing an action through. The method of claim 8,
The learning model includes:
Wherein a combination of a plurality of sensor data filtered for each space is input and a probability for at least one activity according to each space is calculated and output. The method of claim 9,
The user action recognition step may include:
A combination of sensor data filtered in a corresponding space is input to a learning model for the specific space in order to recognize a behavior of a user located in the specific space and a maximum likelihood And recognizing an action of a user located in the specific space by selecting an activity.

Images