JP2012037919A - Information appliance operating system using augmented reality technology - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily and appropriately perform integrated remote control of a plurality of information appliances connected to a network.SOLUTION: An information appliance 14 includes a marker 1. The information appliance operating system includes: a communication unit 13 communicated with the information appliance; a camera 2; an A/D converter 3; a frame memory 4; an image data output unit 5 for outputting image data of the frame memory; a D/A converter 6; a display 7; a marker detecting unit 8 for analyzing the image data for every frame and detecting an ID and a position of the marker; an information acquisition unit 10 acquiring status information and/or content information on the corresponding information appliance when detecting the marker; an image composition unit 9 composing image data of an operation interface including the status information or the content information on the corresponding information appliance with the image data of the frame memory when detecting the marker; and an operation unit 12 transmitting an operation parameter value and/or content selection information input through the operation interface to the corresponding information appliance.

Description

  The present invention relates to an information home appliance operation system using augmented reality technology, and more particularly to a system for remotely operating a plurality of information home appliances connected to a home network using augmented reality technology.

  In recent years, information appliances that can be connected to a network, such as televisions, HDD / DVD recorders, digital video cameras, and the like, are becoming popular. In order to realize an environment in which information appliances from different manufacturers are compatible and can be connected to a home network, users can enjoy digital life comfortably. Guidelines are defined by the Network Alliance (see, for example, Patent Document 1 and Non-Patent Document 1).

In DLNA, the following four types of device classes are mainly defined as types of information appliances connected to the network.
DMS (Digital Media Server): A server function that stores content, distributes the content to clients (DMR or DMP) connected to the network, and plays them back (eg, HDD recorder)
DMR (Digital Media Renderer): A function to display content distributed from DMS (eg, video monitor, audio receiver, etc.)
-DMC (Digital Media Controller): A function that performs control such as searching for content stored in the DMC and distributing it to the DMR (eg, remote control, etc.)
-DMP (Digital Media Player): A function that combines the functions of DMC and DMR (eg, home theater, home game machine, etc.)

  In this way, without complicated settings by the user, DLNA-compliant information home appliances are interconnected to a home network according to a protocol compliant with the DLNA guidelines. For example, as shown in FIG. 6, a single DMC is used to connect to the network. All the DLNA-compatible information home appliances (DMS, DMR1 to DMR3) can be remotely operated in an integrated manner. That is, in the illustrated system, a list of DMRs (DMR1 to DMR3) connected to the network is displayed on the DMC display screen, and a list of contents stored in the DMS is displayed on the DMC display screen. When the user selects a desired DMR and content using the DMC, the selected content is displayed on the selected DMR.

  However, since all DLNA compatible home appliances connected to this network are identified by their network addresses, the DMR list displayed on the DMC display screen identifies the location of each DMR. It does not contain any information to do. For this reason, when the number of DLNA-compliant information home appliances connected to the network increases, the user cannot easily determine where each information home appliance is installed. As a result, what is an information home appliance intended for operation by the DMC? There has been a problem that erroneous operations are likely to occur, such as operating different information appliances.

For example, in FIG. 6, assuming that DMR1 is installed in a bedroom, DMR2 is installed in a dining room, DMR3 is installed in a living room, and the user is in the living room, in this case, the DMR displayed on the display screen of the DMC Since the list does not include information for identifying the location of each DMR, the user can easily select which one of DMR1 to DMR3 can be viewed on the DMC display screen. Although it could not be determined, the DMR 3 in the living room should actually be operated, but an erroneous operation such as operating the DMR 1 in the bedroom was likely to occur.
Similarly, when there are a plurality of operable amplifiers / speakers, when it is desired to increase the volume in the room where the user is, it is not possible to easily determine which amplifier / speaker should be operated on the display screen of the DMC. Misoperations such as operating an amplifier / speaker in another room were likely to occur.

JP 2008-310451 A

"DLNA Networked Device Interoperability Guidelines", expanded: March 2006

  Therefore, an object of the present invention is to enable an integrated remote operation of a plurality of information home appliances connected to a network more easily and accurately.

  In order to solve the above-described problem, according to the first invention, a system for remotely operating a plurality of information home appliances connected to a network using augmented reality technology, the information home appliance is provided, and at least the A marker that displays the MAC address or IP address of an information appliance, a communication unit that is connected to the network and communicates with the information appliance, and a camera that acquires a video image of an area including the marker of the information appliance to be operated An A / D converter that converts the video signal of the camera into a digital signal, a frame memory that stores image data for each frame output from the A / D converter, and a new one in the frame memory An image data output unit that outputs image data of one frame stored first each time image data of a frame is stored; A D / A converter connected to an output terminal of a data output unit, a display for displaying an image based on a video signal output from the D / A converter, and an output from the A / D converter Analyzing the image data for each frame, the marker detection unit for detecting the MAC address or IP address of the marker included in the image data, the position of the marker in the image data, and the marker detection unit When detecting a marker, based on the MAC address or IP address of the marker, an information acquisition unit that acquires status information and / or content information of the corresponding information home appliance through the communication unit, and the marker detection unit At the position of the marker in the image data stored in the frame memory, Information corresponding to the image synthesis unit that synthesizes image data of the operation interface including status information and / or content information, and the operation parameter value and / or content selection information input through the operation interface through the communication unit There is provided a system including an operation unit that transmits to a home appliance.

  According to a preferred embodiment of the first invention, the marker comprises at least one light source and a light source driving unit for blinking the light source, and the MAC address or the IP address is determined according to the blink pattern of the light source. According to another preferred embodiment, the light source comprises an LED.

  According to still another preferred embodiment of the first invention, the marker has a plurality of light sources of different emission colors, and the MAC address or the IP address is divided into a plurality of parts according to the number of the light sources. For each color of the light source, a different part of the MAC address or the IP address is displayed according to the blinking pattern, and the marker detection unit detects the part of the MAC address or the IP address displayed by the light source, respectively. Then, a combination of the detected MAC address or IP address portions in a predetermined order is detected as the MAC address or the IP address.

  According to still another preferred embodiment of the first invention, the marker has three light sources having different emission colors, and the three light sources are arranged at the vertices of a triangle, and are provided in the information home appliance. The marker detection unit includes a reference position information storage unit that stores in advance reference information related to the relative positions of the three light sources on the information home appliance, and the marker detection unit is configured to store the 3 in the image data. Detecting the relative position of each light source, and comparing the detected relative position information with the reference information read from the reference position information storage unit, the distance from the camera to the information home appliance, and the The orientation of the information appliance is detected, and the image composition unit converts the image data of the operation interface according to the distance to the information appliance and the orientation of the information appliance. After it is combined with the image data stored in the frame memory.

  According to still another preferred embodiment of the first invention, the communication unit, the camera, the A / D converter, the frame memory, the image data output unit, the D / A converter, the display, and the marker The detection unit, the information acquisition unit, the image synthesis unit, and the operation unit are incorporated in a single portable casing.

  In order to solve the above-described problem, according to the second invention, a system for remotely operating a plurality of information appliances connected to a network using augmented reality technology, the information appliance is provided, and at least the A marker that displays an ID of an information appliance, a communication unit that is connected to the network and communicates with the information appliance, a camera that acquires a video image of an area including the marker of the information appliance to be operated, and the camera An A / D converter that converts the video signal into a digital signal, a frame memory that stores image data for each frame output from the A / D converter, and a new frame of image data in the frame memory Connected to the output terminal of the image data output unit and the image data output unit that outputs the image data of one frame stored first The D / A converter, the display for displaying an image based on the video signal output from the D / A converter, and the image data for each frame output from the A / D converter are analyzed. A table in which the ID of the marker included in the image data, a marker detection unit for detecting the position of the marker in the image data, and a correspondence relationship between the ID and an address on the network of the information appliance When the marker detection unit detects the marker, the information home appliance corresponding to the marker ID is identified with reference to the table, and the status information and / or content information of the information home appliance is sent to the communication unit. And when the marker detection unit detects the marker, the information acquisition unit to acquire through the image data stored in the frame memory An image synthesizer that synthesizes image data of the operation interface including the status information and / or content information of the corresponding information appliance at the position of the maker, and the operation parameter value or content selection information input through the operation interface or There is provided a system including an operation unit that transmits both of them to the corresponding information home appliance through the communication unit.

  According to a preferred embodiment of the second invention, the marker comprises at least one light source and a light source driving unit for blinking the light source, and the ID is determined by a blinking pattern of the light source, According to another preferred embodiment, the light source comprises an LED.

  According to still another preferred embodiment of the second invention, the marker has a plurality of light sources of different emission colors, and the ID is divided into a plurality of parts according to the number of the light sources, For each color, the different part of the ID is displayed according to the blinking pattern, and the marker detection unit detects the part of the ID displayed by the light source, and combines the detected IDs in a predetermined order. Is detected as the ID.

  According to still another preferred embodiment of the second invention, the marker has three light sources of different emission colors, and the three light sources are arranged at the vertices of a triangle, and are provided for the information home appliance. The marker detection unit includes a reference position information storage unit that stores in advance reference information related to the relative positions of the three light sources on the information home appliance, and the marker detection unit is configured to store the 3 in the image data. Detecting the relative position of each light source, and comparing the detected relative position information with the reference information read from the reference position information storage unit, the distance from the camera to the information home appliance, and the The orientation of the information appliance is detected, and the image composition unit converts the image data of the operation interface according to the distance to the information appliance and the orientation of the information appliance. After it is combined with the image data stored in the frame memory.

  According to still another preferred embodiment of the second invention, the communication unit, the camera, the A / D converter, the frame memory, the image data output unit, the D / A converter, the display, and the marker The detection unit, the table, the information acquisition unit, the image composition unit, and the operation unit are incorporated in a single portable casing.

  According to still another preferred embodiment of the first and second inventions, the network comprises a network complying with Digital Living Network Alliance (DLNA) guidelines.

  According to the present invention, by using augmented reality technology, an information home appliance to be operated is photographed with a camera, the captured video image is displayed on the display, a marker of the information home appliance is detected, and the video image The status information and content information of the information home appliance are displayed together with the operation interface at the position of the information home appliance. Therefore, the user can operate the information home appliance to be operated with the camera and display it on the display, so that the user can operate the information home appliance to be operated with ease while visually checking the information home appliance. Never operate an information appliance in another location.

It is a block diagram which shows the structure of the information home appliance operation system using the augmented reality technology by one Example of this invention. It is a figure which shows one example of the blink pattern of LED. It is a figure which shows the Example of the form of a portable remote control. It is a photograph which shows an example of the video image displayed on the display of the system of this invention. It is a photograph which shows an example of the video image displayed on the display of the system of this invention. It is a figure which shows an example of the home network which interconnected the DLNA corresponding | compatible information household appliance.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram showing a configuration of an information home appliance operation system using augmented reality technology according to one embodiment of the present invention. Referring to FIG. 1, according to the present invention, each of a plurality of information home appliances 14 interconnected to a network 15 is provided with a marker 1 that displays at least the ID of the information home appliance 14.
In this embodiment, the network 15 is a network that complies with the Digital Living Network Alliance (DLNA) guidelines, but may be a different network.

  The ID of the information home appliance 14 may have any configuration as long as each information home appliance 14 can be identified, for example, different numbers, characters, symbols, or the MAC address of the information home appliance 14 or the MAC address. Or an IP address or a part of the IP address.

In this embodiment, the marker 1 includes a single light source 1a and a light source driving unit 1b that blinks the light source 1a, and the ID of the information home appliance 14 is displayed by the blinking pattern of the light source 1a. In this case, it is preferable that the light source 1a consists of LED.
In this way, by configuring the marker 1 from the light source 1a such as the blinking LED, the marker 1 can be detected even in the dark, and it is not necessary to attach a large marker to each information appliance 14 and the aesthetic Will also improve.

  The light source 1a blinks by blinking in a pattern for displaying an ID following blinking indicating preamble information. For example, as shown in FIG. 2, the light source 1a is turned on as “1”, and turned off as “0”, and 1 and 0 are repeated a predetermined number of times, and then “1” is finally sent twice. After the transmission and the preamble information are transmitted, the ID information is transmitted with the blinking pattern P2 displaying the ID.

  The blinking time interval of the light source 1a is determined so as to be optimal in relation to the bit rate of the video image acquired by the camera 2 described later. That is, now, assuming that the bit rate of the video image is 30 fps, the blinking time interval of the light source 1a is determined to be at least 1/30 s or longer. For example, in this embodiment, the frame rate (video image bit rate) at the time of camera shooting is set to 1/30 s, and the light source 1a is set to blink at an interval of 1/15 s, whereby the rate is once every two frames. Thus, the blinking state of the light source 1a is detected. In this case, whether an odd frame or an even frame is to be detected is determined depending on which frame has a blinking state in which preamble information is clearly detected.

  According to the present invention, the communication unit 13 connected to the network 15 and communicating with the information home appliance 14 is provided. The communication unit 13 is preferably wirelessly connected to the network 15. According to the present invention, the camera 2 that acquires the video image of the area including the marker 1 of the information appliance 14 to be operated, the A / D converter 3 that converts the video signal of the camera 2 into a digital signal, and A A frame memory 4 for storing image data for each frame output from the / D converter 3 and each time one new frame of image data is stored in the frame memory 4, the first frame stored in the frame memory 4 is stored. An image output unit 5 for outputting image data, a D / A converter 6 connected to an output terminal of the image output unit 5, and a display for displaying an image based on a video signal output from the D / A converter 6 7 is provided.

  Furthermore, according to the present invention, when the image data for each frame output from the A / D converter 3 is analyzed and the marker 1 is detected in the image data, the ID of the marker 1 and the image data A marker detector 8 for detecting the position of the marker 1 is provided.

  The analysis of the image data by the marker detection unit 8 is performed as follows. First, image data for each frame is converted into monochrome and then compared, and a change point in the image data is extracted using a feature point extraction algorithm in a known image recognition technique. In this case, since the camera may move during the shooting of the video image, predetermined characteristic parts in the image data are detected, and these characteristic parts are overlapped to thereby detect the light source 1a. A blinking change point is detected. When a plurality of detected change points in the image data that are candidates for the marker 1 coincide with the preamble information, the change points are recognized as markers, and the subsequent blinking pattern of the ID is detected.

According to the present invention, when the marker 11 detects the marker 1 when the marker 11 detects the marker 1 and the table 11 that records the correspondence between the ID and the address on the network of the information appliance 14, the marker The information home appliance 14 corresponding to the ID 1 is specified, and the information acquisition unit 10 that acquires the status information and / or content information of the information home appliance 14 through the communication unit 13 and the marker detection unit 8 detects the marker 1. At this time, the image data of the operation interface (see 16 in FIG. 3) including the status information and / or the content information of the corresponding information home appliance 14 is synthesized at the position of the marker in the image data stored in the frame memory 4. An image composition unit 9 is provided.
In this case, when the ID of the information appliance 14 is composed of a MAC address or an IP address, the table 11 is not necessary, and the information acquisition unit 10 can detect the MAC address of the marker detected by the marker detection unit 8 or Based on the IP address, the status information and / or content information of the corresponding information home appliance is acquired through the communication unit 13.

  The status information of the information home appliance 14 is information indicating the operation state of the information home appliance 14, and includes, for example, the current operation mode of the information home appliance 14, the volume level and the sound quality level of the information home appliance 14, and the like. In addition, the content information includes titles, playback times, and the like of videos and music stored in the information home appliance 14.

  Furthermore, according to this invention, the operation part 12 which transmits the operation parameter value input through the operation interface, the content selection information, or both to the corresponding information household appliance 14 through the communication part 13 is provided. The operation unit 12 includes a push button, a joystick, a touch panel incorporated in the display 7, or a combination of two or more thereof.

In this embodiment, all the components except the marker 1 provided in the information home appliance 14 are incorporated into a single portable casing 17 as shown in FIG. 3 to form one remote controller. As a result, it can be easily carried and used in rooms required in the home.
Further, all the components except the marker 1 can be incorporated into a commercially available mobile phone or PDA.

Thus, the user photographs the information home appliance 14 to be operated with the camera 2. At this time, the LED provided in the information home appliance 14 is always photographed. At this time, the LED of the information appliance 14 blinks repeatedly and continuously transmits preamble information and ID.
The video signal of the camera is converted into a digital signal by the A / D converter 3, and image data for each frame output from the A / D converter 3 is stored in the frame memory 4. Each time one new frame of image data is stored in the frame memory 4, the first frame of image data stored by the image data output unit 5 is output to the D / A converter 6 for D / A conversion. Based on the video signal output from the device 6, the captured video image is displayed on the display 7.

  Further, the image data for each frame output from the A / D converter 3 is analyzed by the marker detection unit 8 to detect the marker of the information appliance 14 (the blinking pattern of the light source 1a), and the marker ID and the image data The position of is detected. At this time, the information acquisition unit 10 acquires the status information and / or content information of the information home appliance (information home appliance to be operated) 14 corresponding to the ID through the communication unit 13. Then, the image composition unit 9 synthesizes the image data of the operation interface including the status information and / or the content information of the information home appliance 14 to be operated at the position of the LED in the image data stored in the frame memory 4. . As a result, the operation interface is synthesized and displayed on the video image on the display 7 screen.

And a user inputs the operation parameter and / or content selection information of the information household appliance 14 which should be operated with the operation part 12 through the operation interface displayed on the display 7. FIG. The input operation parameter and / or content selection information is transmitted to the information home appliance 14 to be operated through the communication unit 13, and the operation is completed.
According to the present invention, the user can take an information home appliance 14 to be operated with the camera 2 and display it on the display 7, so that the user can perform remote operation very easily while visually checking the information home appliance to be operated. Therefore, the information home appliance 14 in another place is not accidentally operated.

As mentioned above, although one Example of this invention was described, the structure of this invention is not limited to the above-mentioned Example, A various modification can be created within the description range of a claim.
For example, according to another preferred embodiment, the marker 1 is composed of a plurality of light sources 1a of different emission colors, and the ID of the information appliance 14 is divided into a plurality of parts according to the number of light sources 1a, For each color of the light source 1a, a portion having a different ID is displayed depending on the blinking pattern. And the marker detection part 8 detects the part of ID which the light source 1a each displays, and detects what combined each detected ID in a predetermined order as ID. In this case, the marker detection unit 8 analyzes the image data for each RGB channel by the same method as that applied to the case of a single light source in the above-described embodiment, so that the color and image of each light source 1a are analyzed. The position of each light source 1a in the data and the ID of each light source 1a are detected.

  According to this embodiment, one ID can be divided into a plurality of parts and transmitted simultaneously in parallel, thereby reducing the ID transmission time (for example, with three light sources 1a). If so, the transmission time is shortened to 1/3), and the time from the start of photographing by the camera 2 to the display of the operation interface on the display 7 is shortened. This embodiment is suitable when a MAC address (48 bits) or an IP address (32 bits) is used as the ID.

  According to another preferred embodiment, the marker 1 is composed of three light sources 1a having different emission colors, and the light sources 1a are provided in the information home appliance 14 in an arrangement located at the vertices of a triangle. And the marker detection part 8 has a reference position information storage part which stored beforehand the reference information regarding the relative position of the three light sources 1a on the information household appliance 14. FIG. Furthermore, the marker detection unit 8 detects the relative positions of the three light sources 1a in the image data, and compares the information on the detected relative positions with the reference information read from the reference position information storage unit. The distance to the information home appliance 14 and the direction of the information home appliance 14 with respect to the camera 2 are detected.

In this case, the marker detection unit 8 analyzes the image data for each RGB channel by the same method as that applied to the case of a single light source in the above-described embodiment, so that the color and image of each light source 1a are analyzed. The position of each light source 1a in the data and the ID of each light source 1a are detected, and further the relative position (distance between the light sources) of each light source 1a is detected. Then, based on the comparison between the distance between the light sources of each light source 1a on the information home appliance 14 (reference information) and the distance between the light sources of each light source 1a in the captured image (information on the detected relative position), the information home appliance from the camera 2 The distance to 14 and the orientation of the information appliance 14 with respect to the camera 2 are detected.
Then, the image composition unit 9 performs data conversion on the image data of the operation interface according to the distance to the information home appliance 14 and the direction of the information home appliance 14, and then composes the image data stored in the frame memory.

  According to this embodiment, the operation interface can be displayed on the display in a manner matched with the video of the information home appliance taken by the camera (if the information home appliance does not face the front but is slanted, the corresponding orientation is displayed. The operation interface of the nearby information appliance is displayed larger than the operation interface of the distant information home, etc.), thereby enabling operation with a more realistic feeling while looking at the display 7. Improves.

  Further, instead of configuring the marker 1 from the light source 1a, for example, a marker such as ARToolKit generally used in augmented reality technology may be used as the marker 1.

[Example]
Next, a verification experiment was conducted in order to confirm the effect of the present invention. In the experiment, a DLNA-compatible HDD recorder was used as the DMS, and two DLNA-compatible speakers were used as the DMR, which were interconnected to the network, and two DMRs were installed on the left and right sides of the DMS. Then, a two-dimensional marker for displaying the ID was attached to each of the DMS and the two DMRs.
In addition, a PDA including all the components of the present invention other than the marker is used as a DMC so that it can be wirelessly connected to the network.

  Then, the DMC camera was operated, the DMS and the marker M1 attached thereto were photographed, and the video image I1 was displayed on the DMC display (see FIG. 4A). Then, at the next moment, a list G1 of contents stored in the DMS is synthesized and displayed at the position of the marker M1 of the DMS on the DMC display screen (see FIG. 4B).

  Next, when the right DMR and the marker M2 attached thereto are photographed by the DMC camera, the display of the current volume level is included at the position of the marker M2 of the DMR of the video image I2 displayed on the DMC display. An image G2 including an operation interface for changing the sound volume level is synthesized and displayed (see FIG. 5). Through this operation interface, the volume level of the right DMR could actually be changed. The volume level could be changed in the same way for the left DMR.

DESCRIPTION OF SYMBOLS 1 Marker 1a Light source 1b Light source drive part 2 Camera 3 A / D converter 4 Frame memory 5 Image data output part 6 D / A converter 7 Display 8 Marker detection part 9 Image composition part 10 Information acquisition part 11 Table 12 Operation part 13 Communication unit 14 Information appliance 15 Network 16 Operation interface 17 Casing

Claims (13)

A system for remotely operating a plurality of information appliances connected to a network using augmented reality technology,
A marker that is provided in the information appliance and displays at least the MAC address or IP address of the information appliance;
A communication unit connected to the network and communicating with the information appliance;
A camera that acquires a video image of an area including the marker of the information appliance to be operated;
An A / D converter for converting the video signal of the camera into a digital signal;
A frame memory for storing image data for each frame output from the A / D converter;
An image data output unit that outputs image data of one frame stored first each time new one frame of image data is stored in the frame memory;
A D / A converter connected to an output terminal of the image data output unit;
A display for displaying an image based on the video signal output from the D / A converter;
A marker that analyzes image data for each frame output from the A / D converter and detects the MAC address or IP address of the marker included in the image data and the position of the marker in the image data A detection unit;
When the marker detection unit detects the marker, based on the MAC address or IP address of the marker, an information acquisition unit that acquires status information and / or content information of the corresponding information home appliance through the communication unit;
When the marker detection unit detects the marker, image data of an operation interface including status information and / or content information of the corresponding information appliance at the position of the marker in the image data stored in the frame memory An image composition unit that composes
An operation unit that transmits an operation parameter value and / or content selection information input through the operation interface to a corresponding information home appliance through the communication unit. The marker is
At least one light source;
A light source driving unit for blinking the light source,
The system according to claim 1, wherein the MAC address or the IP address is displayed according to a blinking pattern of the light source.   The system of claim 2, wherein the light source comprises an LED.   The marker has a plurality of light sources of different emission colors, and the MAC address or the IP address is divided into a plurality of parts according to the number of the light sources, and the blinking pattern is determined for each color of the light source. Different portions of the MAC address or the IP address are displayed, and the marker detection unit detects the MAC address or the IP address portion displayed by the light source, and detects the detected MAC address or the IP address portion. The system according to any one of claims 1 to 3, wherein a combination of them in a predetermined order is detected as the MAC address or the IP address.   The marker has three light sources of different emission colors, and the three light sources are provided in the information home appliance in an arrangement located at the vertices of a triangle, respectively, and the marker detection unit is provided on the information home appliance. A reference position information storage unit that stores in advance reference information related to the relative positions of the three light sources, wherein the marker detection unit detects the relative positions of the three light sources in the image data, and detects the detected relative positions; Is compared with the reference information read from the reference position information storage unit to detect the distance from the camera to the information home appliance, and the orientation of the information home appliance with respect to the camera, the image composition unit, The image data of the operation interface is converted according to the distance to the information appliance and the direction of the information appliance, and then the image data stored in the frame memory is converted. A system according to any one of claims 1 to 4, characterized in that the synthesized.   The communication unit, the camera, the A / D converter, the frame memory, the image data output unit, the D / A converter, the display, the marker detection unit, the information acquisition unit, the image synthesis unit, and The system according to claim 1, wherein the operation unit is incorporated in a single portable casing. A system for remotely operating a plurality of information appliances connected to a network using augmented reality technology,
A marker that is provided in the information appliance and displays at least an ID of the information appliance;
A communication unit connected to the network and communicating with the information appliance;
A camera that acquires a video image of an area including the marker of the information appliance to be operated;
An A / D converter for converting the video signal of the camera into a digital signal;
A frame memory for storing image data for each frame output from the A / D converter;
An image data output unit that outputs image data of one frame stored first each time new one frame of image data is stored in the frame memory;
A D / A converter connected to an output terminal of the image data output unit;
A display for displaying an image based on the video signal output from the D / A converter;
A marker detection unit that analyzes image data for each frame output from the A / D converter and detects an ID of the marker included in the image data and a position of the marker in the image data;
A table that records the correspondence between the ID and the address of the information appliance on the network;
When the marker detection unit detects the marker, the information home appliance corresponding to the marker ID is identified with reference to the table, and status information and / or content information of the information home appliance is acquired through the communication unit. An information acquisition unit to
When the marker detection unit detects the marker, image data of an operation interface including status information and / or content information of the corresponding information appliance at the position of the marker in the image data stored in the frame memory An image composition unit that composes
An operation unit that transmits an operation parameter value and / or content selection information input through the operation interface to a corresponding information home appliance through the communication unit. The marker is
At least one light source;
A light source driving unit for blinking the light source,
The system according to claim 7, wherein the ID is determined by a blinking pattern of the light source.   The system of claim 8, wherein the light source comprises an LED.   The marker has a plurality of light sources of different emission colors, and the ID is divided into a plurality of parts according to the number of the light sources, and the parts having different IDs are different depending on the blinking pattern for each color of the light sources. The marker detection unit is displayed, and the marker detection unit detects the portion of the ID displayed by the light source, and detects a combination of the detected IDs in a predetermined order as the ID. Or the system of claim 9.   The marker has three light sources of different emission colors, and the three light sources are provided in the information home appliance in an arrangement located at the vertices of a triangle, respectively, and the marker detection unit is provided on the information home appliance. A reference position information storage unit that stores in advance reference information related to the relative positions of the three light sources, wherein the marker detection unit detects the relative positions of the three light sources in the image data, and detects the detected relative positions; Is compared with the reference information read from the reference position information storage unit to detect the distance from the camera to the information home appliance, and the orientation of the information home appliance with respect to the camera, the image composition unit, The image data of the operation interface is converted according to the distance to the information appliance and the direction of the information appliance, and then the image data stored in the frame memory is converted. A system according to any one of claims 8 to claim 10, characterized in that the synthesized.   The communication unit, the camera, the A / D converter, the frame memory, the image data output unit, the D / A converter, the display, the marker detection unit, the table, the information acquisition unit, the image composition The system according to any one of claims 7 to 11, wherein the unit and the operation unit are incorporated in a single portable casing.   The system according to any one of claims 1 to 12, wherein the network is a network compliant with DLNA (Digital Living Network Alliance) guidelines.