KR101400730B1 - Smart refrigerator and method for controlling smart refrigerator - Google Patents

Abstract

The disclosed technology relates to an intelligent refrigerator and a method of controlling the same, and more particularly, to a refrigerator that automatically recognizes a user of a refrigerator and automatically performs an appropriate operation according to a user, and a method of controlling the same. Among the embodiments, the intelligent refrigerator includes a photographing unit that acquires an image including a user's face; An image processor for extracting facial feature information from the image, comparing the extracted facial feature information with at least one facial feature information stored in advance, and recognizing the user; And a controller for performing an operation corresponding to the recognized user.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an intelligent refrigerator and an intelligent refrigerator,

The disclosed technology relates to an intelligent refrigerator and a method of controlling the same, and more particularly, to a refrigerator that automatically recognizes a user of a refrigerator and automatically performs an appropriate operation according to a user, and a method of controlling the same.

As information technology develops, it becomes possible to intelligently control and manage home appliances such as TVs and refrigerators. For example, Korean Unexamined Patent Application Publication No. 2007-0050234 (a refrigerator equipped with an RFID system and a method for displaying contents thereof) discloses a method for efficiently managing the contents of a refrigerator using an RFID system. As another example, Korean Patent Laid-Open No. 2010-0133895 (method of managing food quality of a smart refrigerator), based on information acquired from acquiring food management information, distribution environment information, and current environmental information from a sensor tag attached to a food, Of the freshness of the product. On the other hand, home appliances such as refrigerators are commonly used by a family living together, and the proper control and management methods may vary depending on the user.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a refrigerator that automatically recognizes a user of a refrigerator and automatically performs an appropriate operation according to a user, and a method of controlling the refrigerator. In particular, the disclosed technology automatically recognizes the user of the refrigerator, thereby limiting the use of the refrigerator according to the user or easily providing information suitable for the user.

According to an aspect of the present invention, there is provided an image processing apparatus comprising: a photographing unit for obtaining an image including a face of a user; An image processor for extracting facial feature information from the image, comparing the extracted facial feature information with at least one facial feature information stored in advance, and recognizing the user; And a controller for performing an operation corresponding to the recognized user.

According to a second aspect of the present invention, there is provided a method of controlling an intelligent refrigerator based on face recognition, comprising: acquiring an image including a user's face; Extracting facial feature information from the image, comparing the extracted facial feature information with at least one facial feature information stored in advance, and recognizing the user; And performing an operation corresponding to the recognized user. The present invention also provides a method of controlling an intelligent refrigerator.

Embodiments of the disclosed technique may have effects that include the following advantages. It should be understood, however, that the scope of the disclosed technology is not to be construed as limited thereby, since the embodiments of the disclosed technology are not meant to include all such embodiments.

According to the disclosed technique, when a person who needs to restrict the use of the refrigerator is approached, the refrigerator door can be locked to prevent the accident caused by the user's weight control, health care, or infant refrigerated access of the cognitive ability Can be effective. In addition, when the use of the refrigerator is not limited, customized food information is provided according to the recognized user, thereby enhancing user convenience.

In addition, according to the disclosed technology, it is possible to more specifically store and manage the food information of the refrigerator using a plurality of RFID reader antennas, and it is also advantageous that the refrigerator can be easily managed by allowing the user to generate RFID tag information .

1 is a block diagram illustrating an intelligent refrigerator according to one embodiment of the disclosed technology.
2 is a view for explaining a structure of an RFID module according to an embodiment of the disclosed technology.
3 is a view for explaining a food storage container according to an embodiment of the disclosed technology.
4 is a flowchart illustrating an intelligent refrigerator control method according to an embodiment of the disclosed technology.
5 is a flowchart illustrating a method of storing food information using an RFID module according to an embodiment of the disclosed technology.

The description of the disclosed technique is merely an example for structural or functional explanation and the scope of the disclosed technology should not be construed as being limited by the embodiments described in the text. That is, the embodiments are to be construed as being variously embodied and having various forms, so that the scope of the disclosed technology should be understood to include equivalents capable of realizing technical ideas.

Meanwhile, the meaning of the terms described in the present application should be understood as follows.

The terms " first ", " second ", and the like are used to distinguish one element from another and should not be limited by these terms. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" to another element, it may be directly connected to the other element, but there may be other elements in between. On the other hand, when an element is referred to as being "directly connected" to another element, it should be understood that there are no other elements in between. On the other hand, other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

It should be understood that the singular " include "or" have "are to be construed as including a stated feature, number, step, operation, component, It is to be understood that the combination is intended to specify that it is present and not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Each step may take place differently from the stated order unless explicitly stated in a specific order in the context. That is, each step may occur in the same order as described, may be performed substantially concurrently, or may be performed in reverse order.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the disclosed technology belongs, unless otherwise defined. Terms such as those defined in commonly used dictionaries should be interpreted to be consistent with the meanings in the context of the relevant art and can not be interpreted as having ideal or overly formal meaning unless explicitly defined in the present application .

The disclosed technology relates to an intelligent refrigerator that is efficiently controlled and managed according to a user. For example, the intelligent refrigerator according to the disclosed technology may include a refrigerator body that is opened at least partially on one side of the refrigerator body to cool the interior of the refrigerator body, and a refrigerator door that is openable and closable in the refrigerator body, And the structure and description of them will be omitted below.

1 is a block diagram illustrating an intelligent refrigerator according to one embodiment of the disclosed technology. 1, the intelligent refrigerator 100 includes a photographing unit 110, an image processing unit 120, a controller 130, an input unit 140, an output unit 150, an RFID module 160, and a storage unit 170 ).

The photographing unit 110 acquires an image including a user's face. According to an embodiment, the photographing unit 110 may be composed of at least one camera including a CCD or a CMOS image sensor. The image acquired by the photographing unit 110 is used as information for recognizing the user of the intelligent refrigerator 100.

The image processing unit 120 extracts facial feature information from the image acquired by the imaging unit 110 and compares the extracted facial feature information with at least one facial feature information stored in advance to recognize the user of the intelligent refrigerator 100 . For example, a process in which the image processing unit 120 recognizes a user according to a predetermined face recognition algorithm is as follows. First, the image processing unit 120 may perform a preprocessing process such as removing the noise of the acquired image or dividing the image. Then, the image processing unit 120 can detect the outline or minutia information of the face in the image obtained based on Principal Component Analysis (PCA), Support Vector Machine (SVM), geometric algorithm and the like. Finally, the image processing unit 120 can recognize the user of the intelligent refrigerator by comparing the information detected in the acquired image with information stored in advance for each user.

The control unit 130 performs an operation corresponding to the user recognized by the image processing unit 120. According to one embodiment, the controller 130 may perform an operation of locking the refrigerator door when the recognized user is the first user. At this time, the first user may be, for example, a baby or an obese patient as a user whose use of the refrigerator is limited due to health, safety, or the like. The control unit 130 can control the locking of the refrigerator door electronically or mechanically. For example, the controller 130 controls the intensity of the electromagnet built in the door of the intelligent refrigerator 100 to lock the door of the intelligent refrigerator 100 for a preset time. In this case, it is preferable that the controller 130 locks the door of the intelligent refrigerator 100 only when the first user is recognized for saving power consumption, and releases the lock after a predetermined time. For example, the controller 130 controls the on / off of a locking module that can mechanically lock the door of the refrigerator, thereby locking the door of the intelligent refrigerator 100.

According to another embodiment, when the recognized user is the second user, the control unit 130 may output the memo previously input through the input unit 140 through the output unit 150. [ The input unit 140 may be implemented as a keyboard, a touch panel, or a microphone for voice recognition, and the output unit 150 may be implemented as a speaker or a display device.

According to another embodiment, the control unit 130 may determine a recommended food based on the information about the recognized user and the food in the intelligent refrigerator 100, and output it. For example, the control unit 130 may determine a recommended food based on at least one of the recognized preferred food, the health status, the combination of the foods in the intelligent refrigerator 100, the shelf life, the current time, and the weather. At this time, the user's favorite food, health condition, etc. may be stored in the intelligent refrigerator 100 in advance.

The RFID module 160 is used to receive and manage the food information in the refrigerator. The RFID module 160 according to an embodiment may include a plurality of RFID antennas and an RFID reader to more specifically manage the positions of the foods. For example, a plurality of RFID reader antennas are installed at equal intervals on each shelf of the refrigerator, and tag identification information is received from an adjacent RFID tag. The structure of the RFID module 160 according to the present embodiment will be described later with reference to FIG.

An RFID tag can be directly attached to food or its packaging during the production and distribution of food. Also, in the case of food without the RFID tag attached thereto, information on the food can be provided using the storage container with the RFID tag attached thereto. For example, after storing kimchi in a storage container, the user can directly input the name, type, manufacturing date, shelf life, etc. of the food into the RFID tag. In this case, when the RFID tag without food information is newly recognized in the received RFID tag identification information, the intelligent refrigerator 100 can receive the food information corresponding to the recognized RFID tag from the user through the input unit 140 .

The storage unit 170 stores information on foods in the refrigerator. According to one embodiment, the storage unit 140 stores the name of the food stored in the refrigerator, the type of the food stored in the refrigerator, the expiration date, the date of storing the refrigerator, and the location in the refrigerator based on the identifiers of the RFID reader antennas that received the tag identification information and the tag identification information Lt; RTI ID = 0.0 > information. ≪ / RTI > Information about the stored food (such as the type of food stored in the refrigerator, location information, shelf life, freshness, etc.) can be output to the screen through the output unit 150.

2 is a view for explaining a structure of an RFID module according to an embodiment of the disclosed technology. The RFID module according to the present embodiment includes a plurality of RFID reader antennas 210 for more accurately recognizing the position of each food item placed in the refrigerator. The plurality of RFID reader antennas 210 may be installed at equal intervals in the refrigerator compartment and the freezer compartment shelf 220 as shown in FIG. Each RFID reader antenna 210 is assigned a unique identifier to provide information on which area of the shelf is located. The RFID reader antenna is connected to the RFID reader built in the refrigerator and a communication cable to transmit the received information.

3 is a view for explaining a food storage container according to an embodiment of the disclosed technology. The RFID tag 320 is attached to the lower end of the food storage container 310 and stores information on the stored food. It may be desirable that the food storage container is sized to match the antenna spacing of FIG.

4 is a flowchart illustrating an intelligent refrigerator control method according to an embodiment of the disclosed technology. The present embodiment relates to a method for controlling the intelligent refrigerator 100 on the basis of face recognition. The user can restrict the use of the refrigerator or provide information suitable for a user according to a user.

In step S410, first, the intelligent refrigerator 100 stores the food information in the refrigerator. The food information may be received from the user through the input unit 140 and stored, or may be stored using the RFID module 160. [ Also, this step may be omitted according to the embodiment. A method of storing food information using the RFID module 160 will be described later with reference to FIG.

In step S420, the intelligent refrigerator 100 acquires an image including the user's face. In step S430, the intelligent refrigerator 100 extracts facial feature information from the image acquired in the previous step, and recognizes the user of the refrigerator by comparing the extracted facial feature information with at least one facial feature information stored in advance. According to the embodiment, the intelligent refrigerator 100 may store the user's facial feature information together with the age, gender, health status, preferred food, restriction on refrigerator usage, etc. of the user.

Thereafter, the intelligent refrigerator 100 performs an operation corresponding to the recognized user as a result of step S430. In one embodiment, the intelligent refrigerator 100 performs an operation of locking the refrigerator door when the recognized user is the first user (S440). For example, the intelligent refrigerator 100 can perform an operation of locking the door of the refrigerator for a preset time by controlling the intensity of the electromagnet built in the door. At this time, the first user may include infants or obese patients whose use of the refrigerator is limited. According to the present embodiment, there is an effect of facilitating the weight control of the obese patient, or preventing infants and infants whose intelligence development is incomplete completely from accidentally ingesting food in the refrigerator or damaging the food or the container.

In another embodiment, when the recognized user is the second user, an operation may be performed to output the memo previously input to the second user (S450). According to the present embodiment, a step (not shown) for inputting a note (for example, " do not drink tomato juice into the refrigerator and go to work ") to the second user is input to the intelligent refrigerator , And when the second user is recognized, the intelligent refrigerator 100 outputs a memo inputted in advance.

In yet another embodiment, the intelligent refrigerator 100 may output a recommendable food determined to be suitable for the user based on the user information or the information about the food in the refrigerator 100, according to the recognized user (S460). For example, the intelligent refrigerator 100 may determine a recommended food based on at least one of the recognized user's gender, age, preferred food, health status, combination of foods in the refrigerator, shelf life, current time, and weather.

5 is a flowchart illustrating a method of storing food information using an RFID module according to an embodiment of the disclosed technology.

In step S510, each RFID reader antenna installed on each shelf of the intelligent refrigerator receives the tag identification information from the adjacent RFID tag. The RFID tag may be a tag attached to a food or a package thereof during production or distribution of food, or may be an RFID tag attached to a separate food storage container. The RFID tag attached to the food storage container may store information on the food directly inputted from the user. For example, a method for a user to input and store food information in an RFID tag is as follows.

If the RFID tag without food information is newly recognized in the tag identification information, the intelligent refrigerator receives the food information corresponding to the newly recognized RFID tag from the user through the input unit (S520). For example, when a new RFID tag is recognized, the intelligent refrigerator can provide a menu item for informing the user of the RFID tag and inputting food information. The user inputs information on the food contained in the food storage container according to the menu items displayed on the screen. The corresponding RFID tag stores the inputted food information in a memory.

In step S530, the RFID module determines the name, type, shelf life, refrigerator storage date, and date of the food in the refrigerator based on the identifiers of the RFID reader antennas received in step S510 or received the tag identification information and the tag identification information input in step S520, And location information in the refrigerator. When the information about the food in the refrigerator is stored, the intelligent refrigerator can display information about the stored food (such as the type of food stored in the refrigerator, the shelf life and freshness) together with the location information of the food. For example, if the freshness of the food is bad, the freshness can be displayed in red, good orange, and good green.

While the system and apparatus disclosed herein have been described with reference to the embodiments shown in the drawings for purposes of clarity of understanding, they are illustrative only and various modifications and equivalent embodiments can be made by those skilled in the art. I will understand that. Accordingly, the true scope of protection of the disclosed technology should be determined by the appended claims.

Claims (17)

A photographing unit for acquiring an image including a user's face;
An image processor for extracting facial feature information from the image, comparing the extracted facial feature information with at least one facial feature information stored in advance, and recognizing the user;
A controller for performing an operation corresponding to the recognized user;
An input unit for receiving a memo to be delivered to a second user;
An output unit for outputting the memo; And
And a locking module for controlling the opening and closing of the door of the refrigerator,
Wherein the control unit controls the locking module to lock the refrigerator door when the recognized user is a first user who is an infant or obese patient,
Wherein the control unit outputs the received memo to the output unit when the recognized user is the second user. The method according to claim 1,
And a storage unit for storing information on foods in the refrigerator,
Wherein the controller performs an operation of outputting a recommendation food based on the information on the recognized user and the stored food. 3. The method of claim 2,
Further comprising a plurality of RFID reader antennas installed on each shelf of the refrigerator and receiving tag identification information from an adjacent RFID tag,
The storage unit stores at least one of the name, type, shelf life, refrigerator storage date, and refrigerator location information of the food stored in the refrigerator based on the identifier of each RFID reader antenna that has received the tag identification information and the tag identification information Intelligent refrigerator. The method of claim 3,
A plurality of storage containers to which RFID tags are attached; And
And an input unit for receiving food information corresponding to the recognized RFID tag from a user when the RFID tag without food information is newly recognized in the tag identification information. 3. The method of claim 2,
Wherein the control unit determines the recommended food based on at least one of the user's preference food, health condition, combination of the stored food, shelf life of the stored food, current time and weather. delete delete The apparatus of claim 1, wherein the control unit
And controlling the intensity of the electromagnet built in the refrigerator door to lock the refrigerator door for a preset time. delete A method for controlling an intelligent refrigerator based on face recognition,
Receiving a memo to be delivered to a second user;
Acquiring an image including a user's face;
Recognizing the user by extracting facial feature information from the image, comparing the extracted facial feature information with at least one facial feature information stored in advance; And
And performing an operation corresponding to the recognized user,
The performing step may include performing an operation of locking the refrigerator door when the recognized user is a first user who is an infant or obesity patient, and outputting the received memo if the recognized user is the second user A method for controlling an intelligent refrigerator. 11. The method of claim 10,
An operation of outputting a recommended food determined based on at least one of the sex, age, preferred food, health status, combination of food in the intelligent refrigerator, shelf life of the food in the intelligent refrigerator, A method of controlling an intelligent refrigerator to perform. 11. The method of claim 10,
Receiving tag identification information from an RFID tag adjacent to each RFID reader antenna installed on each shelf of the intelligent refrigerator; And
Storing at least one of the name, type, shelf life, refrigerator storage date, and refrigerator location information of the food stored in the intelligent refrigerator based on the identifier of each RFID reader antenna that has received the tag identification information and the tag identification information Further comprising the steps of: 13. The method of claim 12,
Further comprising receiving, from a user, food information corresponding to the recognized RFID tag when the RFID tag without food information is newly recognized in the tag identification information. delete delete 11. The method of claim 10, wherein performing
And controlling the intensity of the electromagnet built in the door of the refrigerator to lock the door of the refrigerator for a predetermined period of time. delete

Images