JP2013207520A - Control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide excellent user operability in remotely controlling an electric device.SOLUTION: A control device comprises: video processing means for processing video images of cameras 240A, 240B imaging the inside of a room where one or more electric devices are disposed; display processing means for displaying the video images from the cameras and a result of processing by the video processing means on the same screen; and operation accepting means for accepting an operation of designating a position on the screen. The video processing means includes electric device detecting means for detecting images of the electric devices from the video images of the cameras. The display processing means displays the video images from the cameras and icons corresponding to one or more electric devices of which the images are detected, on the same screen. When the operation accepting means accepts an operation of designating a display position of an icon, a control command is transmitted to the electric device corresponding to the icon.

Description

  The present invention relates to a control device, and more particularly to a control device that remotely controls an electric device that is a controlled object.

  Various UI (user interface) technologies for controlling home appliances have been proposed. For example, in Patent Document 1 (Japanese Patent Laid-Open No. 2000-194467), home appliances such as an air conditioner and a television are handled as objects, a plurality of objects are captured as real images taken with a camera, and the object of interest and its detailed information are displayed simultaneously. On the other hand, an information display device for remote control by sending a command for controlling the object is shown.

JP 2000-194467 A

  When driving and controlling home appliances, there is a demand for providing a user-friendly UI. However, in Patent Document 1, when a real image obtained by photographing a home appliance in a room is displayed, when the user transmits a code for controlling the desired home appliance being displayed, It is necessary to operate a button of the input means 14 provided separately from the display screen, and the operability is not excellent.

  Therefore, an object of the present invention is to provide a control device having excellent user operability when remotely controlling an electric device.

  According to an aspect of the present invention, a control device includes a video processing unit for processing a video of a camera that captures a room in which one or more electric devices are arranged, a video from the camera, and a processing result by the video processing unit. Are displayed on the same screen, and operation accepting means for accepting an operation for designating a position on the screen.

  The video processing means includes electrical device detection means for detecting an image of the one or more electrical devices from the video of the camera, and the display processing means includes the video from the camera and one or more of the images detected. The icon corresponding to the electrical device is displayed on the same screen, and when the operation accepting unit accepts an operation for designating the display position of the icon, a control command is transmitted to the electrical device corresponding to the icon.

  Preferably, the control device further includes means for receiving electrical device information indicating an operation state of the electrical device from one or more electrical devices. When the operation accepting unit accepts an operation for designating the display position of the icon, the display processing unit displays the electric device information of the electric device corresponding to the icon on the same screen.

  Preferably, when the operation accepting unit accepts an operation for designating a display position of the icon and an operation for designating a display position of the electrical equipment information of the electrical equipment corresponding to the icon, the operation acceptance means is addressed to the electrical equipment corresponding to the icon. Then, a control command according to the electric equipment information is transmitted.

  Preferably, the display processing unit displays the electric device information at a position where an image of the electric device in the video is detected.

  Preferably, the display processing unit displays an icon corresponding only to an electric device that has responded to the inquiry among the one or more electric devices.

Preferably, the icon is generated from an image detected for the corresponding electrical device.
Preferably, the operation accepting unit accepts a touch operation for designating a position on the touch panel, inputs information corresponding to the accepted operation, and the touch panel is provided on the screen.

  Preferably, the video display processing means includes a video processing means and a display processing means. When it is determined that a predetermined condition is satisfied, the video display processing means is activated.

  Preferably, when it is determined that the power consumption of one or more electric devices arranged in the room exceeds the target value, the video display processing means is activated.

  A control method according to another aspect of the present invention includes a step of processing a video of a camera that captures an interior of a room in which one or more electric devices are disposed, a video from the camera, and a processing result of the step of processing the video. Are displayed on the same screen, and a step of accepting an operation for designating a position on the screen is provided.

  Processing the video includes detecting an image of one or more electrical devices from the camera video. In the displaying step, an operation for designating an icon display position is displayed in the step of displaying an image from the camera and an icon corresponding to one or more electric devices in which the image is detected on the same screen. When accepted, a control command is transmitted to the electrical device corresponding to the icon.

  ADVANTAGE OF THE INVENTION According to this invention, the outstanding user operativity at the time of remotely controlling an electric equipment can be acquired.

1 is a schematic configuration diagram of a network system according to an embodiment of the present invention. It is a hardware block diagram of the home controller which concerns on embodiment of this invention. It is a figure which shows the table for control which concerns on embodiment of this invention. It is a figure which shows an example of the corresponding | compatible table which concerns on embodiment of this invention. It is a hardware block diagram of the household appliance which concerns on embodiment of this invention. It is a hardware block diagram of the repeater which concerns on embodiment of this invention. It is a functional block diagram of the home controller which concerns on embodiment of this invention. It is a figure which shows the structure of the communication apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the warning screen which concerns on embodiment of this invention. It is a figure which shows the example of a screen of the picked-up image by the camera which concerns on embodiment of this invention. It is a figure which shows the example of a screen for displaying household appliance information with the picked-up image of a camera. It is a process flowchart for the warning process starting based on the power consumption which concerns on embodiment of this invention. It is a flowchart of the warning process which concerns on embodiment of this invention. It is another process flowchart for the warning process starting based on the power consumption which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. It should be noted that the same components are denoted by the same reference symbols in the respective drawings, and detailed description thereof will not be repeated.

  Here, an electric device refers to a device that is driven when electricity is supplied. In particular, a home appliance is illustrated as an electric device in this embodiment. In the present embodiment, the home appliance information is information including types of operation modes of home appliances, data indicating which mode is currently operating, and data indicating power consumption.

<Overall configuration of network system>
FIG. 1 is a schematic configuration diagram of a network system 1 according to an embodiment of the present invention. Referring to FIG. 1, a network system 1 is installed in, for example, a house or an office. Here, a house is assumed. The network system 1 is installed in a living room air conditioner 200A, a kitchen TV (television) 200B, a home appliance such as a microwave oven 200C, a refrigerator 200D, and a lighting 200E, and a camera 240A installed in the living room and a kitchen. Camera 240B.

  The cameras 240A and 240B are, for example, CCD (Charge Coupled Device) cameras, and have a function of photographing the entire installed room and transmitting the photographed video as required.

  The illustrated home appliances are merely examples, and the present invention is not limited thereto, and the type of room is not limited to the living room or the kitchen. Here, air conditioner 200A, television 200B, microwave oven 200C, refrigerator 200D, and lighting 200E may be collectively referred to as home appliances 200A to 200E. The cameras 240A and 240B may also be collectively referred to as cameras.

  The network system 1 includes a solar power generation device 200X, a storage battery 200Y, and a power conditioner 200Z for controlling them. Storage battery 200Y may be installed in a house or the like, or may be a storage battery for automobiles that is also used as a storage battery for houses.

  The power conditioner 200Z supplies power to the storage battery 200Y, the system, and the home appliances 200A to 200E via the power line 402. And the power conditioner 200Z acquires electric power from the solar power generation device 200X, the storage battery 200Y, and a system (such as an electric power system provided by an electric power company) via the power line 402.

  In network system 1 according to the present embodiment, communication device 400A for air conditioner 200A is installed between air conditioner 200A and the outlet of air conditioner 200A. In other words, communication device 400A is plugged into an outlet of air conditioner 200A, and adapter 250A of air conditioner 200A is plugged into communication device 400A. A communication device 400B for the television 200B is installed between the television 200B and the outlet of the television 200B. In other words, communication device 400B is plugged into the outlet of television 200B, and adapter 250B of television 200B is plugged into communication device 400B. Similarly, a communication device 400C for the microwave oven 200C is installed between the microwave oven 200C and the outlet of the microwave oven 200C. In other words, the communication device 400C is plugged into the outlet of the microwave oven 200C, and the adapter 250C of the microwave oven 200C is plugged into the communication device 400C. Communication device 400D for refrigerator 200D is installed between refrigerator 200D and the outlet of refrigerator 200D. In other words, the communication device 400D is plugged into the outlet of the refrigerator 200D, and the adapter 250D of the refrigerator 200D is plugged into the communication device 400D. A communication device 400E for the illumination 200E is installed between the illumination 200E and the outlet of the illumination 200E. In other words, the communication device 400E is plugged into the outlet of the lighting 200E, and the adapter 250E of the lighting 200E is plugged into the communication device 400E.

  The network system 1 includes a home controller 100 corresponding to home appliances 200A to 200E, a solar power generation device 200X, a storage battery 200Y, a power conditioner 200Z for controlling them, a camera, a control device for controlling home appliances, and the like. The network system 1 includes a repeater 300 for relaying communication between the home appliances 200A to 200E and the camera and the home controller 100.

  The home controller 100 can communicate with the repeater 300, the camera, the home appliances 200A to 200E, the solar power generation device 200X, the storage battery 200Y, the power conditioner 200Z, and the like via a wired or wireless network 401. As the network 401, for example, a wireless LAN, ZigBee (registered trademark), Bluetooth (registered trademark), wired LAN (Local Area Network), or PLC (Power Line Communications) is used. The home controller 100 may be a portable tablet terminal, may be detachable from a base placed on a table, or may be fixed to a wall of a room. .

<Hardware configuration and table of home controller 100>
2, home controller 100 includes a memory 101, a display 102, a tablet 103, buttons 104, a communication interface 105, a speaker 107, a clock 108, and a CPU (Central Processing Unit) 110. Input interfaces 1031 and 1041 for accepting user operations on the tablet 103 and the button 104 and an output interface 1021 for outputting information to the display 102 are further provided. As a device that outputs information to the user, a display 102 that displays images and a speaker 107 that outputs sound can be used.

The memory 101 is realized by various types of RAM (Random Access Memory), ROM (Read-Only Memory), a hard disk, and the like. For example, the memory 101 is a USB (Universal Serial Bus) memory, a CD-ROM (Compact Disc-Read Only Memory), a DVD-ROM (Digital Versatile Disk-Read Only Memory), which is used via a reading interface. USB (Universal Serial Bus) memory, memory card, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, MO (Magnetic Optical Disc), MD (Mini Disc), IC (Integrated Circuit) card (excluding memory cards) It is also realized by a medium for storing a program in a nonvolatile manner such as an optical card, a mask ROM, an EPROM, and an EEPROM (Electronically Erasable Programmable Read-Only Memory).

  The memory 101 stores a control program executed by the CPU 110, a target table 305, pattern data 306, an address table 307, and a correspondence table 308, which will be described later, and a display for storing image data to be displayed on the display 102. Including memory. Details of the table will be described later.

  The display 102 displays various information such as the status of the home appliances 200A to 200E and the power conditioner 200Z by being controlled by the CPU 110.

  In the present embodiment, a tablet 103 is laid on the surface of the display 102. That is, in the present embodiment, display 102 and tablet 103 constitute touch panel 106 for accepting user operations. The touch panel 106 detects a user's touch operation and outputs coordinates indicating a touch position on the tablet 103 to the CPU 110.

  The button 104 is disposed on the surface of the home controller 100. The button 104 has a plurality of buttons and keys such as an enter button, a return button, a direction button, and a numeric keypad.

  The CPU 110 accepts a user operation via the tablet 103 or the button 104 and analyzes the accepted operation. When the received operation is an operation related to the operation control of the home appliance, an operation control command according to the operation is generated based on the analysis result. The generated operation control command is transmitted to the target home appliance via the communication interface 105.

  The communication interface 105 transmits / receives data via the network 401 under the control of the CPU 110.

  The speaker 107 outputs sound based on a command from the CPU 110. For example, the CPU 110 causes the speaker 107 to output sound based on the sound data.

  The clock 108 inputs the current date and time to the CPU 110 based on a command from the CPU 110.

  The CPU 110 executes various types of information processing by executing various programs stored in the memory 101. In other words, the processing in the home controller 100 is realized by each hardware and software executed by the CPU 110. Such software may be stored in the memory 101 in advance. The software may be stored in a storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet.

  Such software is read from the storage medium by using a reading device (not shown), or downloaded by using the communication interface 105 and temporarily stored in the memory 101. The CPU 110 stores the software in the form of an executable program in the memory 101 and then executes the program.

  As storage media, CD-ROM (Compact Disc-Read Only Memory), DVD-ROM (Digital Versatile Disk-Read Only Memory), USB (Universal Serial Bus) memory, memory card, FD (Flexible Disk), hard disk , Magnetic tape, cassette tape, MO (Magnetic Optical Disc), MD (Mini Disc), IC (Integrated Circuit) card (excluding memory card), optical card, mask ROM, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory) And the like, for example, a medium for storing the program in a nonvolatile manner.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

  FIG. 3 is a diagram showing a table for control according to the embodiment of the present invention. FIG. 3A shows an example of the target table 305. Referring to FIG. 3A, the target table 305 includes data 305A indicating the room name corresponding to each room in the house and data 305B indicating the target value of the power consumption preset by the user for each room. Stored in association. The target value of the data 305B satisfies the upper limit condition described later.

  FIG. 3B is a diagram showing an example of the address table 307 according to the embodiment of the present invention. Referring to FIG. 4, address table 307 includes data 307 </ b> A indicating the name of the home appliance corresponding to each of the control target home appliances (that is, target home appliances for which power measurement is performed and the measured power is aggregated), Data 307B indicating the name of the room where the home appliance is installed and an address 307C for communicating with the home appliance are included.

  FIG. 4 is a diagram showing an example of the correspondence table 308 according to the embodiment of the present invention. The correspondence table 308 is generated corresponding to each of the cameras 240A and 240B for each room and stored in the memory 101. Although FIG. 4 shows a correspondence table 308 for the kitchen camera 240B, a correspondence table for the living room camera 240A can be similarly generated. The correspondence table 308 is read / written mainly for video display processing and user operation acceptance by the camera.

  Referring to FIG. 4, correspondence table 308 includes data 308 </ b> A indicating the name of the home appliance corresponding to each control target home appliance, data 308 </ b> B indicating the position of the home appliance image in the display image, and a picture representing the appearance of the home appliance. -It includes icon data 308C indicating an image and data 308D indicating the display position of the icon.

<Hardware configuration of TV 200B>
As an example of home appliances, the configuration of a television 200B will be described. Referring to FIG. 5, a television 200B includes a memory 201 for storing data and programs related to driving, a button 204 for accepting an operation for control by a user, a communication interface 205, a display stereo (not shown), An AV output processing unit 207 including an AV (Audio-Visual) signal processing unit for outputting sound and a CPU 210 are included.

  The memory 201 can be realized in the same manner as the memory 101 of the home controller 100. The memory 201 stores a control program executed by the CPU 210, a command input to the television 200B, a current operation mode 201A that is data for indicating the current operation mode of the television 200B, and the like.

  The current operation mode 201A corresponds to household appliance information of the television 200B, and is data for specifying and indicating the current operation mode in a list in which the types of operation modes of the television 200B are arranged in order. .

The communication interface 205 transmits and receives data to and from the home controller 100 via the network 401 under the control of the CPU 210. As described above, the communication interface 205 uses the wireless LAN, ZigBee (registered trademark), Bluetooth (registered trademark), wired LAN (Local Area Network), PLC (Power Line Communications), or the like, so that the repeater 300 is used. Data is transmitted / received to / from the home controller 100.

<Hardware configuration of repeater 300>
FIG. 6 is a hardware configuration diagram of repeater 300 according to the embodiment of the present invention. Referring to FIG. 6, repeater 300 includes a CPU 301, a communication interface 302 for communicating via a network 401, and a storage unit 303.

  In the present embodiment, power consumption information measured for home appliances 200 </ b> A to 200 </ b> E is transmitted to home controller 100 via relay 300. On the home controller 100 side, power consumption (unit: kWh) is calculated for each home appliance.

<Overview of network system operation>
Next, an outline of the operation of the network system 1 according to the present embodiment will be described.

  Home controller 100 receives power consumption data of home appliances 200A to 200E from home appliances 200A to 200E installed in a house via network 401. Each of the home appliances 200 </ b> A to 200 </ b> E includes a communication unit for communicating with the home controller 100.

  The home controller 100 accumulates the received power consumption data of the home appliances 200A to 200E, thereby accumulating the power consumption amounts of the home appliances 200A to 200E over a predetermined period, and stores the accumulated power consumption amount in a predetermined storage area. If necessary, it is output (displayed) to the outside or transmitted to an external information processing apparatus.

  The power conditioner 200Z detects the amount of power generated by the photovoltaic power generation apparatus 200X and the amount of power stored by the remaining charge of the storage battery 200Y using a power sensor (not shown), and supplies the detected power amount to the home controller 100. Send.

  The home controller 100 controls the operation of each home appliance using one or more types of power supplied from the power supplied from the grid, the amount of power generated by the solar power generation device 200X, and the amount of power stored in the storage battery 200Y. In this case, the operation of each home appliance is controlled under an upper limit condition that results in at least power consumption in a range not exceeding the amount of power supplied.

  The home controller 100 normally controls the operation of each home appliance under the upper limit condition. When it is determined that the power consumption of each home appliance exceeds the target for each room under the upper limit condition, a warning is output to the display 102 via the output interface 1021.

  Here, in order to simplify the explanation, it is assumed that a user operation for controlling the operation of each home appliance is received by the input interface 1031 via the operation unit (touch panel 106 or button 104). May be received from an operation unit such as a button 204 inherently included in the.

<Functional configuration>
FIG. 7 is a functional configuration diagram of the home controller 100 according to the embodiment of the present invention. Referring to FIG. 7, the home controller 100 displays a communication unit 310 corresponding to the communication interface 105, an operation receiving unit 311 for receiving user operations, a video processing unit 320 for processing received video signals, and information. A display processing unit 330 for transmitting the request and a request unit 340 for transmitting the request. The communication unit 310 includes processing units such as error correction and modulation / demodulation.

  The video processing unit 320 generates an icon for generating an icon indicating an appearance pattern / image of the home appliances 200A to 200E and the home appliances 200A to 200E for detecting images of the home appliances 200A to 200E from the video received from the communication unit 310. Part 322.

  The display processing unit 330 includes a video display unit 331 for displaying a video (still image or moving image) received via the communication unit 310 on the display 102, a warning display unit 332 for displaying warning information, and an icon generation unit 322. Includes a home appliance icon display unit 333 for displaying the icons of home appliances 200A to 200E generated on the display 102, and a home appliance information display unit 334 for displaying home appliance information acquired from the home appliances 200A to 200E.

  The request unit 340 includes a home appliance information request unit 341 for requesting home appliance information from the home appliances 200A to 200E, and a video request unit 342 for requesting a video from the camera.

(Home appliance detection processing)
The detection process by the household appliance detection part 321 is demonstrated. Home appliance detection unit 321 detects images of home appliances 200 </ b> A to 200 </ b> E using pattern data 306 in memory 101.

  The pattern data 306 includes a feature amount of an image of the home appliance and an icon of the home appliance in association with a home appliance name that is an identifier of various home appliances.

  The home appliance detection unit 321 extracts an image in units of frames from the camera video received from the communication unit 310, and for each partial image in the extracted frame image, the position data 308B and the feature amount in the frame image of the partial image. Is calculated. The calculated feature amount of the partial image is collated with the feature amounts of various home appliances in the pattern data 306, and the image similarity is calculated from the collation result. When it is determined that the calculated similarity for the partial image is greater than a predetermined threshold, the home appliance name data 308A corresponding to the feature amount used for calculating the similarity in the pattern data 306, and the partial image The calculated position data 308B is stored in the correspondence table 308.

  With the above-described home appliance detection process, the position of the home appliance image and the home appliance name in the captured image are detected for each home appliance in the room that is photographed by the camera.

  Here, the size of each partial image in the frame unit image indicates a size empirically determined from the image size of the home appliance. Further, it is assumed that the size and shape of the above-mentioned frame unit image matches that of the display 102, and the shape indicates a rectangle. The above-described position is indicated by, for example, a two-dimensional coordinate value in the rectangular image area.

(Icon generation processing)
The icon generation unit 322 searches the pattern data 306 for each home appliance name indicated by the data 308A detected by the home appliance detection unit 321, and reads the corresponding icon. The read data 308C of each icon is stored in the correspondence table 308 so as to correspond to the home appliance name data 308A.

  In addition, the icon production | generation part 322 is good also as reducing the partial image of the household appliance detected by the household appliance detection process, and using the resulting nail image as an icon.

(Icon display processing)
The home appliance icon display unit 333 reads the home appliance name data 308A corresponding to the icon data 308C generated by the icon generation unit 322 from the correspondence table 308, associates the icon with the home appliance name, and sets a predetermined position (this) Are referred to as icon positions). The home appliance icon display unit 333 stores icon position data 308D in association with the data 308C of each icon displayed on the display 102.

(Household appliance information display processing)
The home appliance information display unit 334 searches the correspondence table 308 based on the user's touch operation position received by the operation reception unit 311, detects the icon position data 308 </ b> D corresponding to the touch operation position, and corresponds to the detected data 308 </ b> D. The image position data 308B is read out. The acquired home appliance information is displayed at the image position of the read data 308B.

  FIG. 8 is a diagram showing a configuration of communication apparatuses 400A to 400E according to the embodiment of the present invention. The communication devices 400A to 400E have the same functional configuration and may be collectively referred to as the communication device 400.

  Referring to FIG. 8, the communication device 400 includes a communication unit 410, an operation control unit 411 for controlling the operation of the home appliance to which the communication device 400 is attached, a request processing unit 440 for processing the received request, A state acquisition unit 450 for acquiring the current operation mode 201A from the home appliance, a power measurement unit 460 for measuring the power consumption of the home appliance, and a power supply unit 470 for supplying power to the home appliance are provided.

<Display screen>
FIG. 9 is a diagram showing an example of a warning screen according to the embodiment of the present invention, which is a screen example displayed on the display 102 by the warning display unit 332.

  FIG. 10 is a diagram showing an example of a screen image captured by the camera according to the embodiment of the present invention, and the screen is displayed on the display 102 by the video display unit 331.

  FIG. 11 is a diagram illustrating an example of a screen for displaying home appliance information together with a video captured by the camera. The screen is displayed on the display 102 by the home appliance icon display unit 333 and the home appliance information display unit 334. In the screen of FIG. 11, the home appliance information IR is displayed only for the television 200 </ b> B for simplification, but actually, the home appliance information IR acquired for each detected home appliance is displayed.

  10 and 11, the value of power consumption (more specifically, current power consumption) included in the home appliance information is displayed in association with the detected home appliance icon.

  The home appliance information IR is information that specifies and indicates the current operation represented by the current operation mode 201A in a list of a plurality of operation modes of home appliances. In order to identify and show, the screen employs aspects such as emphasis and inversion.

  The user can switch the operation mode of the home appliance by operating the touch panel 106. Specifically, the type of operation mode after switching is touched from the operation mode list indicated by the home appliance information of the desired home appliance. The operation reception unit 311 receives the operation and outputs a touch position. The CPU 110 identifies the image position data 308B corresponding to the touch position from the correspondence table 308 based on the touch position from the operation reception unit 311 and reads the home appliance name of the data 308A corresponding to the identified data 308B. Then, the address 307C of the home appliance is read from the address table 307 based on the read home appliance name.

  At this time, the CPU 110 detects that the “i-th” type of operation mode in the operation mode list is touched using the coordinate value of the touch position from the operation reception unit 311 and a predetermined coefficient.

  Thus, when the operation mode switching operation of the home appliance is performed, the CPU 110 outputs the address 307C of the home appliance and “i-th”.

(Calculation of power consumption for each room)
Here, the communication device 400 of each home appliance generates a communication packet storing power consumption data of the corresponding home appliance measured by the power measuring unit 460, and periodically transmits the communication packet to the home controller 100 via the communication unit 410. Send. The address of the transmission source home appliance is stored in the header of this communication packet.

  The CPU 110 of the home controller 100 receives the communication packet via the repeater 300. The CPU 110 searches the address table 307 based on the received communication packet address. Based on the search result, an address 307C that matches the received address is detected. The received power consumption is cumulatively added for each room name indicated by the data 307B corresponding to the detected address 307C. Thereby, the power consumption by the household appliance of the said room is calculated for every room.

<Flowchart>
FIG. 12 is a process flowchart for warning output based on power consumption according to the embodiment of the present invention. Referring to FIG. 12, first, CPU 110 receives power consumption from each home appliance (step S3), and calculates the power consumption for each room according to the above-described procedure (step S5).

  Subsequently, the calculated power consumption for each room is compared with the target data 305B of the power consumption corresponding to the room name data 305A indicated by the data 305B of the target table 305 (step S7). Based on the comparison result, it is determined whether the power consumption of the room exceeds the target value indicated by the data 305B (step S9). When it is determined that the target is not exceeded, that is, the condition of calculated power consumption ≦ target value is satisfied (NO in step S9), the warning image being displayed is deleted from the screen of the display 102 (step S10). Thereafter, the process returns to step S3.

  If the warning image is not displayed, the process of step S10 is passed and the process returns to step S3.

  If it is determined that the target is exceeded, that is, the condition of calculated power consumption> target value is satisfied (YES in step S9), the CPU 110 activates a warning process (step S11). Thereafter, the process returns to step S3.

  In this way, the process of FIG. 12 calculates the total power consumption of the home appliances for each room, and activates the warning process (see FIG. 13) when the calculated total power consumption exceeds the target. The process of FIG. 12 is executed at intervals of 0.2 seconds, for example.

FIG. 13 is a flowchart of the warning process according to the embodiment of the present invention.
Referring to FIG. 13, when the warning process is started by CPU 110, first, warning display unit 332 displays a warning image (see FIG. 9) on display 102 (step S13). The warning image includes information indicating a room name whose power consumption exceeds the target. Here, it is assumed that the power consumption of the kitchen exceeds the target.

  After displaying the warning image, the user touches the “Yes” button in FIG. 9 if the user wishes to control the home appliance so that the power consumption is below the target, but otherwise the “No” button. Touch.

  The operation accepting unit 311 accepts a user button operation and outputs the accepted operation content. If the CPU 110 determines from the output of the operation reception unit 311 that the “No” button has been operated (“No” in Step S15), the process ends.

  On the other hand, if it is determined that the “Yes” button has been operated (“Yes” in Step S15), the display processing unit 330 displays the information on the power consumption for each room calculated in Step S5 on the display 102 (Step S17). ).

  Thereafter, the video request unit 342 of the request unit 340 transmits a video request to the camera 240B using the address of the camera 240B in the kitchen that has been determined that the power consumption has exceeded the target (step S19). It is assumed that the memory 110 stores the address of the camera installed in the room corresponding to each room name.

  Upon receiving the video request, the camera 240B takes a picture of the kitchen when receiving the request and returns the video to the request source. Thereby, the home controller 100 can acquire the image | video of a kitchen (step S21).

  The home appliance detection unit 321 of the video processing unit 320 performs the above-described home appliance detection process from the video of the kitchen (step S23). Based on the detection result, the video processing unit 320 determines whether the home appliance has been detected (step S25). If it is determined that the home appliance has not been detected (NO in step S25), the process proceeds to step S37.

  If it is determined that the home appliance has been detected (YES in step S25), CPU 110 transmits an inquiry communication packet to each home appliance detected by communication unit 310. In the header of this communication packet, the address 307C of the detected home appliance read from the address table 307 is stored as the destination address.

  In each home appliance communication apparatus 400, the communication unit 410 stores a corresponding home appliance address in advance. Therefore, when the communication unit 410 receives the communication packet, the communication unit 410 determines whether or not the received communication packet address is a communication packet addressed to its own home appliance by checking the address of the home appliance stored in advance. If it is determined that it is not addressed to itself, the communication packet is discarded.

  If the communication device 400 of each home appliance determines that the received communication packet is addressed to its own home appliance, it transmits a response to the home controller 100. The response includes the address of the home appliance. CPU110 detects the detection household appliance which can communicate based on whether a response is received, and specifies the said detection household appliance as a control object (step S29).

  As a result, home appliances that do not respond, that is, home appliances that have been unplugged from the communication device 400 and have not received power supply, can be excluded from the control target and removed from the home appliance information acquisition target in the subsequent step 33.

  Subsequently, the home appliance information request unit 341 of the request unit 340 transmits a request for home appliance information to each home appliance identified as a control target (step S31).

  When the communication device 400 receives a request for home appliance information addressed to its own home appliance, the state acquisition unit 450 generates a communication packet of home appliance information from the current operation mode 201 </ b> A of the memory 201. The generated communication packet is transmitted to the home controller 100.

  CPU110 of home controller 100 acquires household appliance information of each household appliance arranged in a kitchen by receiving a communication packet of household appliance information (Step S33).

  Thereafter, an image of the captured video is displayed on the display 102 (step S35). Specifically, the video display unit 331 displays video captured by the camera on the same screen of the display 102, and the home appliance icon display unit 333 simultaneously displays the detected home appliance icon generated by the icon generation unit 322 (see FIG. 10). . In FIG. 10, when the icon of the detected home appliance cannot be displayed on one screen, the icon can be scroll-displayed by operating the scroll button SR. Moreover, the household appliance information display part 334 displays the household appliance information acquired about the detected household appliance on said same screen by the above-mentioned household appliance information display process (refer FIG. 11).

  According to FIG. 11, it is possible to display a captured image of a room where the power consumption of the home appliance exceeds the target, an icon of the home appliance placed in the room, and home appliance information on the same screen. Also, the home appliance information is displayed at the position of the corresponding home appliance image in the video, facilitating the association between each home appliance in the room and the current operation mode.

  When the user touches the icon of the desired home appliance in FIG. 11 via the touch panel 106 and touches the home appliance information of the home appliance to switch to another operation mode, the operation accepting unit 311 accepts the operation.

  After displaying the screen of FIG. 11, the CPU 110 determines whether or not the user has touched the touch panel 106 based on the output from the operation reception unit 311 (step S <b> 37). If it is determined that the touch operation is not performed (“No input (continuation)” in step S37), the determination process in step S37 is repeated.

  On the other hand, when it is determined from the output of the operation reception unit 311 that the “switching” operation mode of the home appliance has been performed (“switching” in step S37), the CPU 110 and the “i-th” address 307C of the home appliance as described above. A communication packet for a mode switching control command including the above is generated (step S39) and transmitted from the communication unit 310 (step S41). Thereafter, the process ends.

  The communication device 400 (in the case of FIG. 11, for example, the communication device 400B of the television 200B) determined to be addressed to itself based on the address 307C of the communication packet, the received communication packet is transmitted via the communication unit 410 to the operation control unit 411. Output to.

  The operation control unit 411 outputs a control command including the “i-th” of the communication packet to the corresponding home appliance. When the corresponding home appliance is the television 200B, the CPU 210 of the television 200B searches the operation mode list of the current operation mode 201A based on the “i-th” control command, and switches the operation mode to the i-th operation mode of the list. Command for output to each unit including the AV output processing unit 207. Thereby, the operation mode of the television 200B is switched to the i-th operation mode, and the current operation mode 201A of the memory 201 is updated to indicate the type of the operation mode after switching. The notification of the completion of the operation mode switching is transmitted to the home controller 100, and the home appliance information IR on the screen of FIG. 11 is updated to indicate the operation mode after the switching.

  Thus, when the power consumption of the home appliances in the kitchen exceeds the target, the operation mode of the TV 200B among the home appliances in the kitchen is turned on by operating the screen of FIG. 11 so that the power consumption is below the target. It can be switched from ON to OFF.

<Other flowcharts>
FIG. 14 is another processing flowchart according to the embodiment of the present invention. The flowchart of FIG. 14 is a modification of the flowchart of FIG. Therefore, in FIG. 14, only the part which deform | transformed from FIG. 13 is demonstrated.

  Referring to FIG. 14, the processes from step S13 to step S23 are performed in the same manner as in FIG.

  In step S25a, it is determined whether or not a home appliance is detected as in step 25. If a home appliance is detected (YES in step 25a), the process proceeds to the process after step S27 described above, but if no home appliance is detected (NO in step S25a), the process for step S51 and subsequent steps is for camera photography. A process for controlling the light source is performed.

  Specifically, the video processing unit 320 extracts a luminance component of the received video, and whether the luminance of the video is a predetermined level from the luminance component, that is, a luminance level at which a partial image of home appliances can be detected. It is determined whether or not (step S51). If it is lower than the predetermined level (YES in step S51), CPU 110 transmits an inquiry communication packet to the room in which the image was taken, that is, the kitchen lighting (step S53). This communication packet stores the address 307C of the kitchen lighting read from the address table 307.

  CPU110 determines whether it can communicate with the said lighting by receiving a response from the lighting 200E of a kitchen (step S55). If it is not determined that communication with the illumination 200E is possible (NO in step S55), the process proceeds to step S37.

  On the other hand, if a response is received from lighting 200E and it is determined that communication is possible (YES in step S55), household appliance information requesting unit 341 transmits a communication packet requesting household appliance information to lighting 200E (step S57).

  The lighting 200E that has received the communication packet of the request reads the current operation mode 201A from the memory 201, and transmits the communication packet stored as home appliance information to the home controller 100 that is the request source.

  CPU 110 determines, based on the home appliance information received from lighting 200E, which operation mode lighting 200E is currently turned off / on (steps S59 and S61). If it is determined to be lit (NO in step 61), the process proceeds to step S37.

  On the other hand, when it is determined that the light is extinguished (YES in step S61), CPU 110 outputs a notification to the outside. This notification is, for example, a sound notification from the speaker 107 (step S63). Then, along with the sound notification, an illumination notification icon is displayed in the display area of the home appliance icon image that is not the display area of the captured video on the display 102 (step S65). Thereafter, the process proceeds to step S37 described above.

  The illumination notification icon is an icon for notifying that the illumination 200E is currently turned off, an icon for prompting the user to turn on the illumination 200E, or the like. By the notification, the user can know that the lighting is turned off in the kitchen and the illumination light quantity for photographing is insufficient.

  The programs according to the flowcharts of FIGS. 12 to 14 described above are stored in the memory 101 and the like, and the functions described above are realized by the CPU 110 reading and executing the programs.

<Modification 1>
The process of FIG. 13 may be started when a predetermined condition is satisfied. That is, in the present embodiment, the predetermined condition is that the power consumption amount of any room exceeds the target value, but the present invention is not limited to this. For example, the predetermined condition may be that a warning or an error is detected based on information received by the home controller 100 from a specific room. In that case, a screen for notifying information of the detected warning / error is displayed.

  The predetermined condition may be that the user operates the touch panel 106 to input an instruction. In that case, processing excluding the warning display (step S13) in FIG. 13 is executed.

<Modification 2>
In this embodiment, the home controller 100 calculates the power consumption of each home appliance, but the calculation of the power consumption may be performed by each home appliance. That is, each home appliance has a power sensor (not shown) that measures its own power consumption in real time, the power consumption measured by the power sensor is cumulatively added using the memory 201, and the cumulative addition result is used as the power consumption amount. It may be configured to transmit to the controller 100. Alternatively, instead of each home appliance, the corresponding communication device 400 may receive power consumption based on the power sensor measurement of the home appliance and calculate the power consumption amount by cumulative addition.

  Alternatively, the repeater 300 may calculate the power consumption. That is, the CPU 301 of the repeater 300 may perform cumulative addition of the power consumption received from each home appliance in the storage unit 303 and transmit the cumulative addition result as the power consumption amount from the communication interface 302 to the home controller 100.

<Modification 3>
In the screen of FIG. 11, the home appliance information IR is acquired and displayed for all the detected home appliances, but the home appliance information IR may be acquired and displayed only for the specified detected home appliance. For example, when the user performs a touch operation on a desired icon image on the screen of FIG. 10, the home appliance information IR may be acquired and displayed only for the detected home appliance of the icon image that has been touched.

<Modification 4>
In the above-described embodiment, the control target home appliance whose operation mode is switched by touching the icon of the detected home appliance is specified, but instead of the icon, the home appliance image in the displayed camera-captured video is touch-operated. The control target home appliance may be specified.

<Modification 5>
In step S35 of FIG. 13, icons are displayed for all detected home appliances, but instead of this, icons may be displayed only for home appliances that have received a response in step S29 among the detected home appliances. Alternatively, icons may be displayed for all detected home appliances, but only a home appliance icon that cannot receive a response in step S29 may be displayed differently from that of other icons.

<Modification 6>
Here, the icon is an image of the appearance of the home appliances 200 </ b> A to 200 </ b> E or a thumbnail image so that the user can easily see, but the icon is not limited thereto. For example, a character icon indicating the name of a home appliance may be used as long as the home appliance can be distinguished by visual recognition.

<Modification 7>
In FIG. 1, the home appliances 200A to 200E communicate with the host computer 100 via the communication devices 400A to 400E, but may communicate with the host computer 00 without using the communication devices 400A to 400E.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  1 network system, 100 home controller, 102 display, 103 tablet, 106 touch panel, 200A air conditioner, 200B TV, 200C microwave oven, 200D refrigerator, 200E lighting, 201A current operation mode, 240A, 240B camera, 305 target table, 306 pattern data 307 Address table, 308 correspondence table, 311 operation accepting unit, 320 video processing unit, 321 home appliance detection unit, 322 icon generation unit, 330 display processing unit, 331 video display unit, 332 warning display unit, 333 home appliance icon display unit, 334 Home Appliance Information Display Unit, 340 Request Unit, 341 Home Appliance Information Request Unit, 342 Video Request Unit, IR Home Appliance Information.

Claims (10)

Video processing means for processing video of a camera that captures an interior of a room in which one or more electrical devices are arranged;
Display processing means for displaying the video from the camera and the processing result by the video processing means on the same screen;
An operation accepting means for accepting an operation for designating a position on the screen,
The video processing means includes an electric device detection means for detecting an image of the one or more electric devices from the video of the camera,
The display processing means includes
Displaying the video from the camera and the icon corresponding to the one or more electrical devices in which the image is detected on the same screen,
A control device that, when the operation accepting unit accepts an operation for designating a display position of the icon, transmits a control command to an electric device corresponding to the icon. Means for receiving electrical equipment information indicating an operating state of the electrical equipment from the one or more electrical equipments;
The display processing means includes
The control device according to claim 1, wherein when the operation accepting unit accepts an operation for designating a display position of the icon, the electrical device information of the electrical device corresponding to the icon is displayed on the same screen.   When the operation accepting unit accepts an operation for designating the display position of the icon and an operation for designating the display position of the electrical equipment information of the electrical equipment corresponding to the icon, it is addressed to the electrical equipment corresponding to the icon The control device according to claim 2, wherein a control command according to the electrical device information is transmitted.   The control device according to claim 2, wherein the display processing unit displays the electrical device information at a position where an image of the electrical device in the video is detected. The display processing means includes
The control device according to any one of claims 1 to 4, wherein an icon corresponding only to an electric device that has responded to an inquiry is displayed among the one or more electric devices.   The control device according to claim 1, wherein the icon is generated from an image detected for the corresponding electric device. The operation receiving means is
Accept the touch operation to specify the position on the touch panel, enter the information corresponding to the accepted operation,
The control device according to claim 1, wherein the touch panel is provided on the screen. Video display processing means including the video processing means and the display processing means;
The control device according to claim 1, wherein when it is determined that a predetermined condition is satisfied, the video display processing unit is activated.   The control device according to claim 8, wherein when the power consumption amount of one or more electrical devices arranged in the room is determined to exceed a target value, the video display processing unit is activated. Processing an image of a camera that captures an interior of a room in which one or more electrical devices are disposed;
Displaying the video from the camera and the processing result of the step of processing the video on the same screen;
Receiving an operation of designating a position on the screen,
Processing the video includes detecting an image of the one or more electrical devices from the video of the camera;
In the displaying step,
Displaying the video from the camera and the icon corresponding to the one or more electrical devices in which the image is detected on the same screen,
In the step of accepting the operation, when an operation for designating a display position of the icon is accepted, a control command is transmitted to an electric device corresponding to the icon.

Images