KR20170054673A

KR20170054673A - Sleeping position verification method using kinect sensors

Info

Publication number: KR20170054673A
Application number: KR1020150157108A
Authority: KR
Inventors: 홍민; 이재훈; 박두순; 민세동
Original assignee: 순천향대학교 산학협력단
Priority date: 2015-11-10
Filing date: 2015-11-10
Publication date: 2017-05-18
Also published as: KR101745794B1

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a sleeping position confirming method using a Kinect sensor capable of confirming an attitude change in a sleep state using a Kinect sensor in a wireless terminal.
The present invention is characterized in that it comprises the steps of: measuring a depth image of a sleeping person and depth information of a human body using a knee sensor 110 including an infrared sensor 111 and a depth sensor 112; Determining a joint position of a sleeping person related to the motion using information measured by the Kinect sensor 110; Confirming a sleeping posture corresponding to the registered sleeping posture when the joint position is confirmed in the image of the sleeping person; Confirming joint movement values by continuously checking the positions of the joints at the time of sleeping; Confirming the converted sleep posture using the joint movement value; Providing a wireless terminal (200) with information on a sleeping image of an identified sleeping person and a positional state of a joint matched to a sleeping image; And displaying images and a sleeping posture of the sleeping posture provided by the wireless terminal 200 on a screen.

Description

[0001] The present invention relates to a sleeping position verification method using kinect sensors,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a sleeping position confirming method using a Kinect sensor capable of confirming an attitude change in a sleep state using a Kinect sensor in a wireless terminal.

It is two-thirds of the total time to wake up and work while a person is alive, and he spends about one-third of his life on the water. In the case of newborn babies, they sleep for 16 hours during 24 hours a day, while for the elderly they sleep for 5 to 6 hours. This 1/3 time is not the time to wastage, but the time when all the functions are organized and recharged to make the remaining 2/3 time more useful. Sleep time is the most important resting time for humans, and it is time to recharge the physical and mental functions. Many studies have suggested various theories about the role of sleep. One is sleeping as an adaptive action to harmonize the activity and relaxation in the external rhythm, and the other is an active physiological function that actively reduces activity in response to the internal environment of the person. In particular, the human body needs a lot of energy in order to maintain an appropriate body temperature. If there is a problem in the sleeping, this energy production function may cause problems. Therefore, sleep as a prerequisite for maintaining body temperature can be seen as a high level of vital function.

A variety of sleep monitoring systems have been developed with the increasing need for a low-cost, efficient pressure and temperature sensor to apply to actual sleep mats to detect the sleep status of the sleepers and to provide a method for healthy lifetime with sensed data. The existing sleep state recognition technology is mainly based on the system that uses this value by measuring the distribution of the pressure generated on the contact surface between the object and the person due to the weight of the person. However, this does not have a continuous monitoring function, This is inconvenient.

In addition, it is possible to confirm the movement of the body during sleep by attaching a sensor directly to the human body, but there is a disadvantage that accurate sleep state can not be confirmed because the sleeping environment is different from the usual state due to the attachment of sensors and the like.

The background art of the present invention is disclosed in the Korean Intellectual Property Office (KIPO) Patent Publication No. 10-2008-0068483 on May 3, 2010.

SUMMARY OF THE INVENTION It is an object of the present invention to solve the above problems and provide a sleeping position confirmation method using a Kinect sensor capable of confirming an attitude change in a sleep state in a wireless terminal using a Kinect sensor.

It is another object of the present invention to provide a method of identifying a sleeping posture using a Kinect sensor capable of confirming a joint of a sleeping person and confirming a change of a sleeping posture by a change of a joint.

It is still another object of the present invention to provide a method for identifying a sleeping posture using a Kinect sensor that can confirm a sleeping environment by confirming a sleeping posture that changes according to environmental requirements of temperature and humidity.

The sleeping position determining apparatus 100 may be configured to detect a sleeping position using a Kinect sensor 110 including an infrared sensor 111 and a depth sensor 112. [ Measuring depth information of an image and a human body; Determining a joint position of a sleeping person related to the motion using information measured by the Kinect sensor 110; Confirming a sleeping posture corresponding to the registered sleeping posture when the joint position is confirmed in the image of the sleeping person; Confirming joint movement values by continuously checking the positions of the joints at the time of sleeping; Confirming the converted sleep posture using the joint movement value; The sleeping position determining apparatus 100 may provide the wireless terminal 200 with the information of the sleeping image of the identified sleeping person and the positional state of the joint matched to the sleeping image; The wireless terminal 200 displays a video and a sleeping posture of the provided sleeping posture on a screen.

Preferably, the sleeping position determining apparatus 100 measures the temperature and humidity of a sleeping place and provides the measured temperature and humidity to the wireless terminal. The wireless terminal is further characterized by displaying the sleep environment for temperature and humidity together.

The sleeping posture change method using the Kinect sensor according to the present invention is advantageous in that the quality of the sleeping person can be confirmed by the user through the wireless terminal.

In addition, since the sleeping posture is analyzed by analyzing the sleeping person 's joint, there is an advantage that the accurate sleeping posture can be known.

In addition, since the sleeping posture changes according to the environmental requirements of temperature and humidity, it is possible to confirm the optimal environmental requirements.

FIG. 1 is a flowchart illustrating an operation procedure of a sleeping position confirmation method using a Kinect sensor according to the present invention.
2 is an exemplary view of an image measured by a depth sensor and an infrared sensor of a Kinect sensor.
FIG. 3 is an exemplary view showing a joint in an image measured by the Kinect sensor according to the present invention; FIG.
Figure 4 is an illustration of a sleeping posture based on boundary information according to the present invention;
Figure 5 is an illustration of a sleeping posture according to the present invention;
FIG. 6 is an exemplary view showing sleep information displayed on a wireless terminal of the present invention; FIG.
7 is a block diagram of a method of determining a sleeping posture using a Kinect sensor according to the present invention.

The details of other embodiments are included in the detailed description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

A description will be made of an operation process of the sleeping position determination method using the Kinect sensor according to the present invention with reference to the drawings.

First, a sleeping person's position information and a human body depth information are measured in a sleeping position determining apparatus 100 provided with a Kinect sensor 110 installed at a sleeping place and including an infrared sensor 111 and a depth sensor 112 Step S101). That is, as shown in FIG. 2, infrared rays reflected from the human body of the sleeping person are sensed through the infrared sensor 111 of the Kinect sensor 110, and the sleeping person's image can be confirmed on a pixel-by-pixel basis using the pixel information , The depth information can be confirmed through the reflected infrared ray made by the COMS sensor using the infrared projector through the depth sensor 112. [ At this time, the depth sensor 112 can obtain 32-bit RGB information having a resolution of up to 640 X 480. The Kinect sensor 110 of the present invention uses the Kinect Nui API to detect skeleton information of a person and detect depth information of a sleeping person in a dark place and depth information of a sleeping person in a sleeping position It becomes possible to measure in three dimensions.

Further, in order to confirm the effect of the sleeping environment, the temperature is measured through the temperature sensor 161 installed at the sleeping place, and the humidity is measured through the humidity sensor 162 (step S102).

Based on the information measured by the Kinect sensor 110, the joint position of the sleeping person related to the motion is three-dimensionally identified and matched to the sleeping image (step S103). For example, as shown in Fig. 3, since the movement of the water surface is mainly caused by the limbs, the head, left shoulder, left elbow, left wrist, left hand, left hip, left knee, left ankle, Y, Z three-dimensional position with respect to joint positions such as right shoulder, right elbow, right wrist, right hand, right hip, right knee, right ankle, right foot and right thumb.

Then, if the joint position is confirmed in the image of the sleeping person, the sleeping posture which is compared with the previously registered sleeping posture is confirmed (step S104). As shown in FIG. 3, the registered sleeping posture can be classified into three types: Foetus, Log, Yearner, Soldier, Freefaller, Starfish ) And the like.

In addition, the sleeping position can be confirmed based on a line connecting the joint position information as shown in FIG. In addition, as shown in FIG. 5, the sleeping posture can be confirmed based on the boundary information among the joint positions of the sleeping person together with the left hand, the left knee, the right hand, and the right knee. Here, the boundary information includes an intermediate line that is an extension line between the neck joint and the vertebra joint, an upper line that is perpendicular to the middle line and is formed based on the central vertebra joint, and a spine base joint that is perpendicular to the middle line. As shown in FIG.

Then, in order to confirm the sleeping person's confrontation during the sleeping, the position of the joint at the time of sleeping is confirmed and accumulated (step S105). Then, the converted sleep posture is confirmed by using the joint movement value (step S106). Thus, it is possible to continuously check whether the sleeping person has moved during sleep. At this time, the motion values of all major joints are calculated every 0.5 seconds, and the calculation of joint motion values is calculated by Equation (1).

[Equation 1]

Here, SM (power saving movement) is the distance of the coordinate movement before the coordinates of the 19 joints in the current coordinate, X _i c is the current X position, and X _i p is the previous X position in the image sequence.

On the other hand, information on the sleeping image of the sleeping person identified by the sleeping position determining apparatus 100, information on the positional state of the joint matched to the sleeping image, and information on temperature and humidity are provided to the wireless terminal 200 such as a sleeping person Step S107). At this time, the communication with the wireless terminal is preferably a local area network such as beacon, NFC, or Bluetooth.

The wireless terminal 200 receives and stores information about the sleeping position provided from the sleeping position determining apparatus 100 and displays it on the screen so that the sleeping person can confirm the sleeping position as shown in FIG. ). At this time, the sleeping person will be able to identify the type corresponding to the sleeping posture and the sleeping posture which change in a time-wise manner. In addition, the sleeping environment for temperature and humidity is also displayed, so that the effect of the sleeping environment on the sleeping posture can be confirmed.

As shown in FIG. 7, the system for realizing the sleeping position confirming method using the Kinect sensor of the present invention includes a sleeping position determining device 100 for accurately determining a sleeping position of a sleeping person using a Kinect sensor, And a wireless terminal 200 that receives information about the sleeping position from the confirmation device 100 and displays the information so that the sleeping person can confirm the sleeping position.

The sleeping position checking apparatus 100 includes a knee sensor 110, a sleeping position registering unit 120, a joint position checking unit 130, a sleeping position checking unit 140, a turning check unit 150, 160, a sleeping person management unit 170, and a communication unit 180.

The Kinect sensor 110 includes an infrared sensor 111 and a depth sensor 112 and three-dimensionally measures a sleeping posture of a sleeping person. The Kinect sensor 110 is installed at a height of 0.5 m to 4.5 m so that the sleeping position can be confirmed. The infrared sensor 111 of the Kinect sensor 110 senses infrared rays reflected from the human body of the sleeping person and provides motion of the sleeping person in units of pixels by using the pixel information and the depth sensor 112 is a COMS Depth information can be obtained through the reflected infrared light produced by the sensor.

The sleeping position registering unit 120 registers and stores information on the sleeping position. The registered silver sleeping posture can be classified into three types as shown in FIG. 3: Foetus, Log, Yearner, Soldier, Freefaller, Starfish, etc. .

The joint position confirmation unit 130 uses the data extracted from the depth sensor 112 to three-dimensionally confirm the joint position of the sleeping person related to the motion. The joint position confirmation unit 130 can check the X, Y, and Z positions of the limb joint related to the movement of the water surface using the depth information of the depth sensor 112. [ At this time, the joints of the limbs include the head, left shoulder, left elbow, left wrist, left hand, left hip, left knee, left ankle, left foot, left thumb, right shoulder, right elbow, right wrist, , Right ankle, right foot, and right thumb. The joint position confirmation unit 130 preferably checks and stores the joint position every 0.5 seconds so as to detect continuous movement of the sleeping person.

The sleeping position confirming unit 140 compares the joint position information of the sleeping state identified by the joint position confirming unit 130 with the previously registered sleeping position to confirm the matching sleeping position. As shown in FIG. 4, the sleeping position confirming unit 140 may check the sleeping position using the line connecting the joint position information confirmed by the joint position determining unit 130, As described above, the sleeping position can be confirmed using the boundary information among the joint positions of the sleeping person along with the left hand, the left knee, the right hand, and the right knee.

The turnover confirmation unit 150 calculates a joint movement value at the time of sleep by confirming the front and back positions of the joint identified by the joint position confirmation unit 130 and confirms the converted sleep orientation using the joint movement value. It is desirable that motion values of all major joints are calculated and accumulated every 0.5 seconds. It is possible to continuously check whether or not the movement has occurred during the sleep by the return check unit 150. [

The sleeping person management unit 170 receives and stores information on the sleeping person's name, sex, age, height, weight, medical history, and the like. This association can be confirmed statistically by the sleep management department.

The sleeping environment verifying unit 160 checks the temperature and humidity of the sleeping environment. The temperature and the humidity can be confirmed by using the temperature sensor 161 and the humidity sensor 162.

The communication unit 180 receives the sleep image confirmed by the Kinect sensor 110, the joint position confirmed by the joint position confirmation unit 130, the sleep position confirmed by the sleep position determination unit 140, And provides temperature and humidity information to the wireless terminal 200 of the sleeping person. At this time, the communication unit 180 preferably communicates with the sleeping wireless terminal through a local communication network such as beacon, NFC, or Bluetooth.

In addition, the wireless terminal 200 of the present invention receives information on the sleeping posture provided from the sleeping position determining apparatus 100 and displays it so that the sleeping person can confirm it. The wireless terminal 200 includes a sleeping information storage unit 210 and a sleeping position display unit 220.

The sleeping information storage unit 210 receives information on the sleeping position provided from the sleeping position determining apparatus 100, and stores the received information in the sleeping information DB 211. The sleeping information storage unit 210 sequentially stores the image of the sleeping position, the type of the sleeping position, and the sleeping environment.

The sleeping posture display unit 220 displays images and a sleeping posture of the stored sleeping posture on the screen through the sleeping information storage unit 210. [ At this time, as shown in FIG. 6, the sleeping person can identify the type corresponding to the sleeping posture and the sleeping posture, which change in a time series. In addition, the sleeping environment for temperature and humidity is also displayed, so that the effect of the sleeping environment on the sleeping posture can be confirmed.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It will be easy to understand. If the practice of such improvement, alteration, substitution or addition is within the scope of the following claims, the technical idea is also deemed to belong to the present invention.

100: Sleeping position determination device 110: Kinect sensor
111: Infrared sensor 112: Depth sensor
120: sleeping position registration unit 130: joint position confirmation unit
140: Sleeping position confirmation unit 150:
160: sleeping environment confirmation unit 170: sleeping person management unit
180: communication unit 200: wireless terminal
210: sleep surface information storage unit 220: sleep surface posture display unit

Claims

Measuring a sleeping person's image and a depth information of a human body with a Kinect sensor including an infrared sensor and a depth sensor;
Confirming a joint position of a sleeping person related to the movement using information measured by the Kinect sensor;
Confirming a sleeping posture that is matched with a previously registered sleeping posture when the joint position is confirmed in the image of the sleeping person;
Confirming joint movement values by continuously checking the positions of the joints at the time of sleeping;
Confirming the converted sleep position using the joint movement value;
Providing a wireless terminal with information about a sleeping image of the identified sleeping person and a positional state of the joint matched to the sleeping image;
And displaying the image and the sleeping posture of the sleeping posture provided by the wireless terminal on a screen.

The method of claim 1 wherein the joint position is selected from the group consisting of head, left shoulder, left elbow, left wrist, left hand, left hip, left knee, left ankle, left foot, left thumb, right shoulder, right elbow, right wrist, Wherein the right knee, the right ankle, the right foot, and the right thumb are included.

The sleeping posture according to claim 1, characterized in that the sleeping posture is any one of Foetus, Log, Yearner, Soldier, Freefaller, Starfish A method of identifying sleeping position using a Kinect sensor.

The method according to claim 1, wherein the sleeping position is confirmed based on a line connecting joint information of the knee.

The sleeping posture according to claim 1, wherein the sleeping posture includes an intermediate line that is an extension line between the neck and vertebra joints in the joint position of the sleeping person along with the left hand, the left knee, the right hand, and the right knee, Wherein the confirmation is made based on boundary information including an upper line formed on the basis of the base line and a lower line formed on the basis of the spine base joint perpendicular to the middle line.

[3] The sleeping position determining apparatus of claim 1, further comprising: measuring temperature and humidity of the sleeping place and providing the same to the wireless terminal; Wherein the wireless terminal further includes a step of displaying a sleep environment for temperature and humidity as well.