JPH10215388A - Remote controller - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a remote controller with excellent operability by easily specifying a control object. SOLUTION: In a control side station, a video camera picks up an image of an actual condition of controlled object devices 11, 12, 13 connected to an I/O controller and transmits the image to an operation side station. All the received images are displayed on an I/O map generating window 161 in the operation side station, and a control object table representing the control content controllable to each controlled object device is displayed. The user designates an area in which the controlled object devices displayed on the all displayed images by two points P1, P2 on a diagonal line and then the I/O map is prepared by designating to which the control object device the area corresponds from the control object table. After that, when the control operation is made to each control object device, the controlled object device is confirmed, designated and designation of the control to the device is attained easily while observing an actual image displayed on an I/O control screen.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote control device, and more particularly to a remote control device for remotely controlling various controls such as on / off of a device to be controlled such as a television.

[0002]

2. Description of the Related Art There is a remote control system capable of remotely controlling various controls on electric appliances using communication. FIG. 8 shows a system configuration of a conventional remote operation system. As shown in FIG. 8A,
The remote operation system includes a control side station a and an operation side operation station b. The control station a is a station that performs actual control of electric appliances, and includes an I / O control device d to which each device to be controlled such as a television c and an air conditioner is connected, and a mobile phone e. The I / O control device d controls a power supply and the like via an I / O control line according to a control target and control contents transmitted from a remote operation side station b via a telephone line by a mobile phone e. . On the other hand, the operating station 2 is mainly composed of a personal computer f having a modem and a mobile phone g. Then, on the display screen of the personal computer f, as shown in FIG. 8B, the names and operation contents of the control target devices connected to the I / O control device are displayed, and are displayed by a predetermined method. The control operation is performed by selecting.

[0003]

However, what is displayed on the display screen of the personal computer f to select the device to be controlled is the name (character) of the device to be controlled as shown in FIG. Or an icon that abstracts the object. For this reason, the conventional operation side station 2 has a screen display far from the actual situation due to the installation state of the control target device and the relationship with other devices,
When there are a plurality of control target devices, there is a problem that it is difficult to find a desired device. In addition, the icon display makes it easier to recognize and specify the control target device, but it is difficult to find the control target device because the positional relationship with other devices is different from the actual one.
In particular, when there are a plurality of control target devices of the same type, for example, when two TVs are connected, even if the TV can be recognized by the icon display, it is possible to determine which TV corresponds to which icon. It was not enough to recognize.

Accordingly, the present invention has been made to solve such a problem, and an object of the present invention is to provide a remote control device which easily specifies a control target and has good operability.

[0005]

According to the first aspect of the present invention, there is provided an image data acquiring means for acquiring image data, an image display means for displaying an image based on the image data acquired by the image data acquiring means, and Area designating means for designating an area on the image displayed by the image display means, correspondence means for associating the area designated by the area designating means with a predetermined control signal, and an image displayed on the image display means A position specifying means for specifying the upper position and an area specified by the area specifying means including the position specified by the position specifying means, and the specified area is associated with the corresponding means by the corresponding means. The above object is achieved by providing a remote control device with control signal output means for outputting the control signal. According to the second aspect of the present invention, there is provided an image data acquiring means for acquiring image data, an image display means for displaying an image based on the image data acquired by the image data acquiring means, and an image displayed on the image displayed by the image display means. Area designating means for designating an area, correspondence means for associating the area designated by the area designating means with a predetermined device to be controlled, and position designation for designating a position on an image displayed on the image display means. Means, and an area specified by the area specifying means including a position specified by the position specifying means, and a plurality of control target devices associated with the specified area by the corresponding means. Control content display means for displaying control content; control selection means for selecting one control content from a plurality of control content displayed on the control content display means; It is and a control signal output means for outputting a control signal corresponding to the control content to the remote operation apparatus to achieve the above purpose. According to a third aspect of the present invention, in the remote control device according to the first or second aspect,
The image acquisition unit acquires image data obtained by capturing an image of the device to be controlled, and the control signal output unit outputs the control signal to a control device that performs various controls on the device to be controlled.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a remote control device according to the present invention will be described below in detail with reference to FIGS. (1) Overview of Embodiment In the present invention, an actual situation of various control target devices connected to an I / O control device is imaged by an imaging device of a control station and transmitted to an operation station. The operation side station displays the received image data on the display device, and performs mapping of the control target device displayed on the display screen. That is, the display area of each control target device is specified on the screen, and the control target device corresponding to the specified area is specified to perform the I / O map creation processing. Thereafter, when performing a control operation on each control target device,
When a device to be controlled is specified while looking at an actual image displayed on the / O control screen, an operation panel showing the contents of control that can be performed on the specified device to be controlled is displayed on the screen. When a necessary control is designated by the user from the operation panel, an I / O control signal indicating the control target device and the control content is transmitted to the control station, and the remote operation is performed. In addition, by using a camera that can be remotely controlled, the control side station can perform mapping using images in a wide range of imaging.

(2) Details of Embodiment FIG. 1 shows a system configuration of a remote operation system using the remote operation device of the present embodiment. The remote control system includes a control station 1 for actually controlling a control target device and an operation station 2 for remotely controlling the control station 1. The control side station 1 mainly controls the control target device, and captures the control target device with a video camera and transmits the image. The operation side station 2 mainly performs remote operation of a video camera, remote operation of an apparatus to be controlled by I / O control, and creation of an I / O map for I / O control. This remote control system is used, for example, for individuals and offices, and by monitoring transmitted images,
It can also be used as a monitoring system in banks and department stores. When the remote control system is used as a monitoring system, various devices such as crime prevention and disaster prevention are used as control target devices.

The control station 1 mainly includes a video camera 4, a control unit 5, an I / O control device 6, a mobile phone 7, and a modem 8. The video camera 4 can be connected to and disconnected from the I / O control device 6 with a cable for NTSC. The video camera 4 constantly captures images and converts image data based on NTSC signals to, for example, 3
It is supplied to the I / O control device 6 periodically at intervals of up to 5 seconds. Although the video camera 4 captures not only moving images but also still images, the video camera 4 is connected to the control unit 5 by using a digital camera or the like that captures only still images. Is also good. The control unit 5 can be connected to the I / O control device 6 via an RS232C cable. The control unit 5 controls the imaging angle of the video camera 4 (upward / downward angle = tilt, left / right direction angle = pan) and zooming according to a camera position change command supplied from the operation side station 2 via the I / O control device 6. The imaging range, focus, and the like are controlled.

The I / O control device 6 has an I / O control line terminal 9 to which various types of control target devices are connected, and performs various controls on the connected control target devices. In the present embodiment, in the example shown in FIG. 1, a lighting switch 13 is connected to the I / O 1 of the I / O control line terminal 9 as a control target device, and the television 12 is connected to the I / O control line terminal 9.
The air conditioner 11 is connected to the I / O3. As the control content of each control target device, in addition to power on / off control, control of a set temperature for the air conditioner 11 can be performed.

Although not shown, the I / O control device 6 includes a CPU for performing various controls, a ROM in which programs for various controls by the CPU are stored, a RAM used as a working area, A compression unit for compressing image data captured by the camera 4 and a memory (flash memory, S-RAM, etc.) for storing image data and the like
Various connection terminals for connecting a video camera 4, a control unit 5, a mobile phone 7, a modem 8, a power cable, and the like;
Other devices are provided. CP of I / O control device 6
There are various types of control executed in the U. Mainly, control of each control target device connected by an I / O control line and image data for use in monitoring and mapping of the control target device in the operation side station. There is a control related to transmission and reception of data.

The I / O control device 6 transmits a control target and control target data representing the control contents to the operation side station 2 as control of the control target device. A signal is received, and control according to the content is performed on the control target device. That is, the I / O control device 6 includes the I / O number of the I / O control line terminal 9, the explicit information of the control target device connected to the I / O control line, and the control content for the control target device. Is transmitted to the operation-side station 2. In the example illustrated in FIG. 1, as the control target data, “I / O1: lighting: power on / off,
"I / O2: TV: power on / off, I / O3: air conditioner: power on / off: temperature Up / Down" are transmitted. The explicit information of the control target devices such as lighting, television, and air conditioner is input by the user of the control station 1 from an input device (not shown), but the device numbers of various devices to be connected in advance are explicitly indicated. May be set in advance, and the connected control target device may be specified by designating the device number. And I / O
When, for example, “I / O3, temperature Up” is transmitted from the operation side station 2 as a control signal, a process of increasing the temperature of the corresponding air conditioner is performed.

On the other hand, transmission and reception of image data by the I / O control device 6 are performed as follows. The I / O control device 6 receives the camera position change command from the operation side station 2 and supplies it to the control unit 5, and converts the image data supplied from the video camera 4 from the NTSC signal to the RGB signal by the compression unit. Then, it is compressed by a predetermined method to obtain compressed image data. The I / O control device 6 adds the position information (pan, tilt, zoom value) of the camera to the header of the compressed image data and transmits the data to the operation-side station 2.

The transmission of the compressed image data is performed through the side of the portable telephone 7 and the modem 8 to which the communication line between the control station 2 is connected. For compression of image data in the I / O control device 6, various existing compression methods can be adopted. For example, JP-A-5-22
The data is compressed by the FST compression method described in Japanese Patent No. 7547. FST compression encodes an A / D converted video signal into a matrix block consisting of a plurality of pixels and a bitmap representing one of two types of reference values,
Removes inter-block and inter-frame redundancy by finding, identifying and coding the non-redundant information in the coded blocks, and comparing each block to the corresponding block in the immediately preceding frame; Further, the amount of information is compressed by coding the current color value in the form of a difference from the immediately preceding color value. Also, as other image compression methods, well-known MPEG and JPEG,
Various methods such as TIM compression described in 154261 may be used.

TIM compression is a method of image compression in which decompression processing of compressed image data can be easily performed on the image data receiving device side. In image compression, TIM compression has the following characteristics: (1) the human eye is sensitive to luminance changes but insensitive to color changes; and (2) is often used when pixels constituting an image are appropriately blocked. The image compression is performed based on the assumption that there are usually only two colors in one block which is generally established in the natural image. That is, each pixel data is divided into image data composed of a luminance signal Y and color difference signals U and V, and block data composed of n × m (usually 4 × 4) is generated. Then, two colors of a first color component (y1, u1, v1) and a second color component (y2, u2, v2) having a luminance signal and a color difference signal, which represent the generated block, and a first color component and a second color component are represented. Data is compressed by generating a block component consisting of an n × m bitmap indicating the distribution of two color components. Then, the generated block component is coded from the block component, a similar block having the same or most similar block component, and the code data indicating the similar block and the difference data with respect to the block component of the similar block. By doing so, the data is further compressed and transmitted to the image data receiving apparatus. The image data compression employed in this embodiment is performed by the I / O control device 6.
The processing is performed by an image compression board (printed board) arranged in, but compression may be performed by software processing such as software MPEG or JPEG.

On the other hand, the operating station 2 is a portable telephone 2 for connecting a telephone line with the control station 1.
0 and remote operation or I / O control of the controlled device by I / O control.
It comprises a personal computer 21 (hereinafter simply referred to as a personal computer 21) that performs various processes such as I / O map creation for O control. The mobile phone 20 and the mobile phone 7 of the control station 1 are mainly connected to each other via a digital mobile communication network at 9600 bps or 19200 bps.
Although connection is made by high-speed communication of ps, another communication method may be used. For example, ISDN (Integrat
(Ed Services Digital Network) or various data communication networks such as a normal telephone line network, the Internet, a FAX communication network, a PHS, a wireless satellite communication, and an optical communication. Also, various telephones can be used for the mobile phones 20 and 7 on both stations, and different types of telephones may be used. Further, regarding the communication speed between the stations 1 and 2, the mobile phones 20 and 7, the communication control device 28,
The setting may be automatically made according to the communication capability of the modem 8.

FIG. 2 shows the configuration of the personal computer 21. As shown in FIG. 2, the personal computer 21 includes a control unit 22 for controlling the whole. This control unit 22
A keyboard 23, a mouse 24, a display device 25, a storage device 26,
The recording medium driving device 27, the communication control device 28, and the input / output IF 29 are connected. The control unit 22 includes a CPU
221, a ROM 222, and a RAM 223.
The ROM 222 is a read-only memory in which various data and programs for performing control by the CPU 221 are stored in advance. The RAM 223 is a random access memory used as a working memory by the CPU 221. When the personal computer 21 is used as the remote control device of this embodiment, the RAM 223 stores various data such as I / O map data, a full screen file, a main image file, a camera position change command, and a camera type. The area is secured.

The keyboard 23 has various keys such as a kana key and a numeric keypad for inputting a kana character, a function key for executing various functions, a cursor key, and the like. As the display device 25, for example, a CRT or a liquid crystal display is used. The display device 25 has a main image window for displaying an image captured by the video camera 4,
A camera control window or the like for remotely controlling the imaging position of the video camera 4, the zoom value, and the like is displayed. The camera control window displays keys for camera control and the entire screen that can be picked up by the video camera. The main image window and the camera control window can be simultaneously displayed, or only one of them can be displayed. In the case of simultaneous display, both windows can be displayed side by side or overlaid.

The mouse 24 is a pointing device, and is an input device for designating a corresponding function by left-clicking a key, an icon, or the like displayed on the display device 25. In the present embodiment, the mouse 24 is used,
When an O control screen is created, an area is specified for an image displayed on the display device, thereby mapping the area to various control target devices. Further, the user instructs capture and synthesis of all images with the mouse 24, designates the imaging position of the video camera 4 by dragging the camera frame displayed in the entire image, and changes the size of the camera frame.

The storage device 26 comprises a readable and writable recording medium and a drive device for reading and writing various information such as programs and data on the storage medium.
As a storage medium used for the storage device 26, a hard disk is mainly used, but a readable and writable recording medium among recording media used in a recording medium driving device 27 described later may be used. . The storage device 26 stores an I / O map data file created by an I / O map creation process for creating an I / O control screen.
Also, the type of the video camera 4 that can be connected to the I / O control device 6 and information (zoom, pan, tilt range, and the like) on items that can be controlled by each video camera are stored. Also,
A dial number for connecting to the control station 1 and all images once captured and combined are stored for each control station. The program storage area of the storage device 26 stores O
In addition to a basic program such as S, various application programs such as an I / O control processing program, an I / O map creation processing program, and an image data reception-related program according to the present embodiment are stored. In the operation side station 2 of the present embodiment, a program for receiving the compressed image data from the control side station 1 and expanding the compressed image data by a method corresponding to the image compression method by the control side station 1 is stored in the storage device 26. Is stored in For example, the I / O of the control station 1
When the control device 6 performs image compression by MPEG, the storage device 26 stores a program for performing decompression by MPEG. As described above, in the image data receiving apparatus of the present embodiment, it is not necessary to set special hardware for special image expansion in the operation side station 2.

The recording medium drive 27 is a drive for the CPU 221 to read a computer program from an external recording medium. The computer program recorded on the recording medium includes the personal computer 2 of the present embodiment.
1 includes programs for various processes executed by the server 1 and dictionaries and data used therein. Here, the recording medium refers to a recording medium on which a computer program is recorded, and specifically, a floppy disk,
Magnetic storage media such as hard disks and magnetic tapes, semiconductor storage media such as memory chips and IC cards, CD-ROMs
, MO, PD (phase change rewritable optical disk) and other storage media from which information can be read optically, paper cards and tapes, and paper such as printed materials for reading programs using character recognition devices (and paper equivalents) Media with the ability to
And other recording media on which computer programs are recorded by various methods. As a recording medium used in the personal computer 21 of the present embodiment, a CD-ROM or a floppy disk is mainly used. The recording medium driving device 27 can read a computer program from these various recording media, and can also write data in the RAM 223 and the storage device 26 when the recording medium is a writable recording medium such as a floppy disk. .

In the personal computer 21, the CPU 221 reads a computer program from an external recording medium set in the recording medium driving device 27 and stores (installs) it in the storage device 26. When the image data reception-related program according to the present embodiment is executed, the program is read from the storage device 16 into the RAM 223 and executed. However, it is also possible to read the data directly from the external recording medium into the RAM 223 by the recording medium driving device 27 instead of the storage device 26 and execute the same. Also, depending on the personal computer 21, the image data reception related program of the present embodiment may be stored in the ROM 222 in advance.
, And this may be executed by the CPU 221.

The communication control device 28 has a connection terminal for the mobile phone 20 and transmits a camera position change command for remotely controlling the video camera 4 and receives compressed image data from the control station 1. It is supposed to do. The communication control device 28 can transmit and receive various data such as text data and bitmap data to and from another personal computer or the like. The input / output IF 29 includes a printing device that prints various images such as a main image, an entire image, and an enlarged image displayed on the display device 25, characters and other various data, and a speaker that outputs audio and music. This is an interface for connecting and disconnecting. As the printing device, various printing devices such as a laser printer, a dot printer, an ink jet printer, a page printer, a thermal printer, and a thermal transfer printer are used.

Next, among operations of the remote control system configured as described above, first, an operation of transmitting and receiving image data obtained by imaging each control target device will be described. FIG. 3 illustrates a display screen when image data is transmitted and received among display screens of the display device 25 according to the present embodiment.
As shown in the figure, the display screen of the display device 25 includes a main image window 151 in which an image captured by the video camera 4 is displayed, and a camera for remotely controlling the imaging position and zoom value of the video camera 4. Control window 152
Is displayed. As will be described later, the camera control window 152 has a full screen 157 in which the entire range that can be captured by combining images captured at each position of the video camera 4 is displayed as a full image.

In the entire screen 157, the video camera 4
A camera frame 158 for instructing and displaying the imaging position and range of the camera is displayed. By changing the position of the camera frame 158 with the mouse 24, the imaging position (pan, tilt) of the video camera 4 is changed, and by changing the size of the camera frame 15, the zoom value of the video camera 4 is changed. You. The imaging position and the zoom value specified by the camera frame 158 in this way are transmitted to the control station 1 and
It is supplied from the / O control device 6 to the control unit 5. The control unit 5 controls the video camera 4 so that the supplied imaging position and zoom value are obtained, so that the video camera 4 captures a detailed image corresponding to the frame specified by the camera frame 158. The image data captured by the video camera 4 is transmitted to the operation side station 2 as a main image, and is displayed in the main image window 151 of the display device 25. On the entire screen 157, all the images captured in a divided manner are continuously displayed as they are, that is, the past images that do not change unless particularly re-acquired are continuously displayed. On the other hand, the main image displayed in the main image window 151 is
Although there is some time lag due to communication, the current image captured by the video camera 4 of the control side station 1 is displayed, and the main image displayed by moving the camera frame 158 of the operation side station 2 also changes. Will do. When the remote control device according to the present embodiment is used in a monitoring system, the periphery of the control station 1 is monitored by moving the position of the main image.

On the left side of the drawing of the camera control window 152, a “capture full screen” key 1 is provided as a camera control key.
53, a “detailed setting” key 154, a “setting stop” key 155, and a “setting transmission” key 156.
All images displayed on the full screen 157 are 1
Since it is not possible to take an image every time, an entire image is taken for each divided screen composed of a plurality of small areas, and each divided screen is synthesized and displayed.

An "all screen capture" key 153 of the camera control window 152 is a key for instructing capture of all images displayed on the full screen 157. "Detailed setting" key 15
Reference numeral 4 denotes a key for setting the brightness level and hue level of the video camera 4, manual operation of focus, and other detailed contents. When this key is left-clicked, the camera control window 152 expands downward. And each setting item is displayed. The “cancel setting” key 155 is a key for canceling the change operation when left-clicked after changing the position or range of the camera frame 158 in the entire screen 157.
Canceling the change operation using this key is effective only before the “setting transmission screen” key is left-clicked. The “setting transmission” key 156 is a key for requesting the I / O control device 6 to change the imaging position and range by the video camera 4.

FIG. 4 explains each divided screen when all images are taken. As shown in FIG. 4, the full screen 157 is divided into divided screens 157a to 157l, and images captured for each divided screen are combined and sequentially displayed on the full screen 157. At this time, the exposure and the hue of each screen may be adjusted to make it easier to see. The movable angle of the pan of the video camera 4 is θp, the movable angle of the tilt is θt, the position of the video camera 4 at the upper leftmost position is the origin (θp = θt = 0 °), and the zoom value is the widest angle. The horizontal angle of view of the video camera 4 is αp,
If the vertical angle of view is αt, the horizontal number of the divided screen is x from the left, and the vertical number is y from the upper side (x = 3, y = 2 for the divided screen 157i), pan and tilt for each divided screen Are calculated by the control unit 22 according to the following equations (1) and (2). Pan coordinates = (x−1) αp (1) Tilt coordinates = (y−1) αt (2) The control unit 11 transmits the calculated data of the pan and tilt coordinates of the divided screen to the video camera 4 sequentially. Is transmitted to the I / O control device 6 together with the position change command of the video camera 4.
Is remotely operated to receive the image at the corresponding division position and synthesize it as an entire image. Note that the number of divisions in the horizontal direction of all images is (θp / αp) +1, and the number of divisions in the vertical direction is (θt /
αt) +1. When the I / O control device 6 supplies the received pan and tilt coordinates to the control unit 5, the control unit 5 moves the video camera 4 to the corresponding coordinate position.

Next, the operation of capturing all images will be described. The control unit 22 of the personal computer 21 first
Mobile phone 7 connected to I / O control device 11 through
Is read out from the storage device 26 and automatically dialed to establish a connection to the communication line. If there are a plurality of control-side stations 1 to be remotely controlled by the personal computer 21, information such as the name, installation location, and dial number of each station is read out from the storage device 26 and displayed on the display device 25, and displayed to the user. Prompt for choice. Then, the control unit 22 transmits the selected control station 1
Automatically dial the mobile phone 7 to connect the communication line. The connection of the communication line with the I / O control device 6 is transmitted to the dial number input by the user from the mobile phone 20 or the keyboard 23 when the manual dial is selected by the user.

After the connection of the communication line, the control unit 22 inquires the type of the video camera 4 to the I / O control device 6 in the following procedure. First, the control unit 22 sends a command to ask the type and version of the camera to the I / O control device 6. Upon receiving this command, the I / O control device 6 checks the type and version of the camera via the control unit 5 and transmits the same to the personal computer 21. When the controller 22 receives the answer of the type and version of the camera, the control unit 22 similarly issues a command for asking a pan range, a command for asking a tilt range, and a command for asking a zoom range of the video camera 4, sequentially performing I / O for each of them. An inquiry is made to the control device 6. When the inquiry about all of the camera type, version, pan range, tilt range, and zoom range is completed, the control unit 22
Stores these in the camera type area of the RAM 223 as camera type information.

Next, the control unit 22 accesses the storage device 26 and, in step 11, the control station 1 connected to the line.
If all the images corresponding to are already present as all image files, all the images corresponding to the connected control side station 1 are read from the all image files in the storage device 26, and are read from the RAM.
223 and the display device 2
5 is displayed on the entire screen 157.

When all the images do not exist in the storage device 26, the control section 22 performs a process of sequentially taking in all the images from a plurality of divided screens. That is, when the “full screen capture” key is selected by the user, the control unit 22 calculates the coordinates of pan and tilt when capturing the entire image from the angle of view of the camera. For example, in the example shown in FIG.
The coordinates of pan and tilt with respect to each of a to 157l are calculated by the above equations (1) and (2),
23.

Next, the control unit 22 sends a command to move the video camera 4 to the corresponding division position of all images to the I / O control device 6. With this command, video camera 4
Is changed (pan, tilt). Then, the control unit 22 receives, from the I / O control device 6, the image data of the divided image captured at the designated position (pan, tilt). Whether or not the divided image is at the designated position is confirmed from the position information of the video camera 4 added to the header of the image data.
Then, the control unit 22 expands (decodes) the received image data of the divided image by a method corresponding to the compression method in the I / O control device 6 and converts the image data into bitmap data.
The image is stored in the corresponding division position of all image files in the RAM 223. After the reception, the divided images stored in the RAM 223 are sequentially displayed at corresponding positions on the entire screen 157. When all the images have been completely taken, the bitmap data of the entire image stored in the RAM 223 is stored in the all image file of the storage device 26 in correspondence with the currently connected control station 1.

Next, the operation of creating an I / O map based on all the images thus captured will be described. This I / O map creation processing is performed by an I / O map for controlling the control target device of the control station 1 with good operability.
This is a process for creating an O control screen. FIG. 5 is a flowchart showing details of the I / O map creation processing. First, the control unit 22 of the personal computer 21 reads out the dial number of the mobile phone 7 connected to the control side station 1 from the storage device 26 via the mobile phone 20 connected to the communication control device 28, and automatically dials. Establishes (connects) a communication line between both stations. However, when there are a plurality of control-side stations 1 to be remotely controlled by the personal computer 21, information such as the name, installation location, and dial number of each station is read from the storage device 26 and displayed on the display device 25, Prompt the user for a selection. Then, the control unit 22 automatically dials the mobile phone 7 of the selected control side station 1 and connects the communication line (step 11). Note that the connection of the communication line with the I / O control device 6 is established by the user when the user selects the manual dialing.
Dials the dial number entered from and connects the communication line between both stations.

After the connection of the communication line, the control section 22 of the personal computer 21 requests the control station 1 to transmit the data to be controlled (step 12). And the control unit 2
2 receives the data to be controlled,
/ O map data file (step 1
3). Then, it is checked whether or not all the images captured by the video camera 4 of the selected control side station 1 are stored in all the image files of the storage device 26 (step 1).
4) If not stored (; N), the above-described full screen fetch process is performed and stored in all image files in the storage device 26.

Then, the control unit 22 reads out all the images corresponding to the selected control station 1 from all the image files in the storage device 26 and writes them in all the image files in the RAM 223, and then outputs the I / O to the display device 25. An O-map creation window is opened to display all images (step 16).
Further, the control target data is read from the I / O map data file in the RAM 223, and the control target table is displayed on the same screen as all the images (step 17).

FIG. 6 shows an I / O map creation window displayed on the display device 25 in the I / O map creation processing. As shown in this figure, the I / O map creation window 161 has a full screen display unit 162,
A control target table display section 163 is provided. On the full screen display unit 162, each device to be controlled controlled by the I / O control station 6 of the control station 1 shown in FIG. 1 is imaged and displayed. The user of the operation side station 2 designates the area for each control target device displayed on the all image display unit 162 by two points P1 and p2 located diagonally by the mouse cursor 164, and determines the designated area. Specify the area for the device to be controlled. The control target device is specified by the I / O in the control target table.
This is performed by designating the number, the name of the device to be controlled, or the frame representing the control content with the mouse cursor 164. Alternatively, the control target device may be designated by inputting a number displayed on the I / O number from a keyboard.

In response to the above operation by the user, the control unit 22 monitors whether or not an area has been designated (step 1).
8) The coordinates of the designated two points P1 and P2 are calculated. 2
With respect to the coordinates of the points P1 and P2, the X and Y coordinates (x pixel, y pixel) with respect to the bitmap of the entire image are calculated with the point O at the upper left of the entire image as the origin (step 19). Then, it is monitored whether or not the device to be controlled for the designated area is designated by the user (step 20). If designated (; Y), the coordinate data of the two points P1 and P2 calculated in step 19 is designated. Is stored in the I / O map data file of the RAM 223 as control target data for the control target device (step 21). The above steps 18 to 21 are performed by the OK key 165, the cancel key 16
6 without selecting (Step 22; N, Step 2
3; N), for each device to be controlled, and finally, OK
When the key is selected (Step 22; Y), the control unit 22
Stores the control target data (including the coordinate data of P1 and P2 specifying the area for each control device) stored in the I / O map data file of the RAM 22 in the storage device 2
6 is stored in the I / O map data file (step 24), the I / O map creation processing ends, and the process returns to the main routine. Further, a cancel key 166 is displayed during the process of specifying the area and the control target for each control target device.
Is selected without storing the contents of the I / O map data file in the RAM 22 in the storage device 26.
The / O map creation processing ends, and the process returns to the main routine.

On the basis of the I / O map created as described above, the I / O map
The operation of performing the / O control operation in the operation side station 2 will be described below. FIG. 7 shows an I / O control window displayed on the display device 25 in the I / O control operation. When an I / O control operation is selected from a predetermined function key or a display screen of the display device 25, the control unit 22
The I / O control window 171 is opened, and all the images read from all the image files in the storage device 26 and the control target data read from the I / O map file are stored in the all image file and the I / O map data file in the RAM 223. At the same time, all the read images are stored in the I / O control window 1
71 is displayed. The user looks at each device to be controlled displayed in the I / O control window 171 while watching the I / O control device.
Mouse cursor 164 on the control target device for performing the control operation
Move and select. This selection operation of the control target device
In the above-described I / O map creation processing, if the mouse cursor 164 is located within the area specified by the two points P1 and P2 corresponding to each device to be controlled, the selection is valid. When a position on the screen is designated by the mouse cursor 164, the control unit 22 displays the I / O control window 1
The position coordinates of the mouse cursor 164 with respect to the origin O of all the images displayed at 71 are confirmed, and the position coordinates and the coordinate data (P1, P2) for each area of the I / O map data are checked.
From 2), the control target device specified by the user is specified.

When the control target device selected by the user is specified, the control unit 22 reads out the control target data for the selected control target device from the I / O map data file, and as shown in FIG. An operation panel 168 is displayed around the display position of the selected device to be controlled. In the example shown in FIG. 7, the area for the air conditioner is specified, and the corresponding operation panel 168 displays the control target device name “air conditioner”, the control contents “power supply: ON, OFF”, and “temperature: Up, Down”. Is displayed. And the user
By selecting one of the operation keys “ON”, “OFF”, “Up”, and “Down” with the mouse cursor 164, the I / O control operation is completed.

When an operation key on the operation panel is selected by the user, the control section 22 reads out data indicating the corresponding I / O number and control contents from the I / O map data file in the RAM 223, and performs I / O control. The signal is transmitted to the control station 1 via the communication control device 28 and the mobile phone 20 as a signal. For example, when the power ON key of the air conditioner is selected by the user, the control unit 22
An O number “3” and data indicating power ON are transmitted to the control station 1 as an I / O control signal.

When the I / O control signal is received by the portable telephone 7 at the control station 1, the I / O control device 6
Supplies a control signal to the control target device connected via the I / O control line of the I / O number, and performs control corresponding to the requested operation from the operation side station 2. As a result, if the content of the I / O control signal indicates that the I / O number is “3” and the power is on, for example, the power of the air conditioner 11 is turned on.

As described above, according to the present embodiment, the control target is not specified by the abstracted icons and characters, but is specified by the actual image of each control target machine supply. This makes it possible to easily recognize the controlled device to be controlled, and to provide a remote control device with good operability. Further, since an actual image is used, the target device can be easily recognized from the positional relationship between the control target devices. Especially when there are multiple devices of the same type, the target control target device is recognized from the positional relationship,
It becomes easy to specify.

In the above-described embodiment, an I / O map is created by specifying an area defined by two points P1 and P2 for each control target device. However, a detailed I / O map for each operation key may be used. . For example, in the I / O map creation window of FIG.
1 and P2, and then the control target table display section 16
Operation key "O" of control object "lighting" (I / O number 1) of No. 3
By specifying "N", a detailed I / O map for each operation key is created. Similarly, the image area of the lighting 13 “OFF”, the air conditioner 11 “ON”, “OFF”, “▲”,
A detailed I / O map for each operation unit is created by designating each image area of “▼” and designating operation keys of the control target table. As described above, when the I / O control is performed on the control target device for which the detailed I / O map for each operation unit is created, the operation panel 168 shown in the I / O control window 171 in FIG. 7 is displayed. Not done. The operation panel 168 is displayed only for the control target device for which the I / O map at the control target device level has been created as described in the embodiment. The I / O control for a control target device for which an I / O map for each operation unit is created is performed when an operation position of the device is designated, for example, the lighting 1
It becomes effective when the mouse cursor 164 is specified in the image area specified for “ON” of No. 3. In this case, the control unit 22 specifies the position specified by the mouse cursor 164, and stores the area including the position in the RAM 223.
By checking the I / O map data file, the control target device and the control content specified by the user are specified, and the corresponding I / O control information is transmitted to the control station 1.

For the same device to be controlled, an I / O map may be created by both area designation of the device and area designation for each operation unit of the device.
For example, in the all image display unit 162 of the I / O map creation window 161, the image area of the entire air conditioner 11 is designated by two points P1 and P2, and then the I / O number of the control target table is set. The I / O map for the air conditioner 13 is created by designating “3” or the part of the control target “air conditioner”. Thereafter, each of the image areas “ON”, “OFF”, “▲”, “▼” of the air conditioner 11 is designated, and the operation keys “ON” key, “OF”
By specifying the "F" key, the "Up" key, and the "Down" key, a detailed I / O map for each operation unit is also created. As described above, when an I / O map is created for both the entire device area and the operation unit area for the same control target device, the control unit 22 responds to the I / O control operation by: It works as follows. That is, the control unit 22 specifies the position specified by the mouse cursor 164, and if the position is within the area specified for the operation unit of the control target device, immediately specifies the position without displaying the operation panel 168. The I / O control information corresponding to the controlled device and the control content is transmitted to the control-side station 1. On the other hand, when the position specified by the mouse cursor 164 is within the area of the entire control target device and outside the region of each operation unit, the operation panel 168 for the control target device is moved to the I / O control window 171. And prompts the user to specify a specific operation key.

When creating a detailed I / O map for each operation unit of the control target device, operability may be deteriorated. For example, a video deck is connected to the I / O control line terminal 9 of the I / O control device 6 of the control side station 1. In addition to turning on / off the power supply, playback, stop, fast forward, rewind, pause, When various video operations such as recording can be controlled, if these I / O maps for each video operation are created from all images,
Since the image of the operation button of the video operation becomes smaller, the operability in the I / O map creation processing and the I / O control processing deteriorates. Therefore, an enlarged image obtained by enlarging a specific area in all the images is displayed in the I / O map creation window 1.
The image may be displayed on the all-image display unit 162 and the I / O control window 171, and the I / O map creation processing may be performed using the enlarged image. Here, as for the request and reception of the enlarged image, as described with reference to FIG. 3, by moving and resizing the camera frame 158 of the camera control window 152, the transmission of the more detailed main image to the control side station 1 is performed. The processing is performed in the same manner as when the requested and received main image is displayed in the main image window 151.

That is, the user selects a predetermined function key for the process of acquiring an enlarged image, or
For example, “OK” is displayed in the O map creation window 161.
An “enlarge” key is arranged on the left side of the key 165, and the “enlarge” key is selected. Then, both windows 16
The camera frame 158 (see FIG. 3) is included in all the images 1 and 171.
Similarly, the enlarged frame is displayed. The user changes the position and size of the enlargement frame to change the area to be enlarged and displayed in the same manner as the main image for the camera frame 158, and then selects the “OK” key 165. Then, the control unit calculates the imaging position (pan, tilt) and zoom value of the video camera 4 from the position and size of the enlargement frame, and transmits them to the control-side station 1. The control station 1 controls the video camera 4 in accordance with the received imaging position and zoom value to capture an image, and transmits the captured image to the operation side station 2 as an enlarged image. Upon receiving the enlarged image, the operation side station 2 displays the enlarged image instead of all the images displayed in both windows 161 and 172. FIG.
In the I / O map creation window 161 shown in (1), the lower part of the enlarged image is hidden by the control target table display unit 163. Therefore, when displaying the enlarged image, the control target table and the "enlarge" key are displayed. , “OK” key and “Cancel” key are displayed as reduced as possible. On the other hand, the I / O control window 171 displays an enlarged image on the entire window.

The user uses the enlarged images displayed in the windows 161 and 171 to specify an area in the I / O map creation process and a position in the I / O control operation. In this case, the control unit 22 determines that the position in the enlarged image designated by the mouse 146 is the control side station 1
Is converted into a position in all images from both the imaging position and zoom value data transmitted to
The control target device and the control content in the O control are specified.

In the embodiment described above, the case where the control target data is received from the control station 1 has been described, but it is not always necessary that the control target data is received. For example, data indicating each control target device and its control (operation) content is determined in advance between the stations 1 and 2, and a control is performed on each area designated by two points P 1 and P 2 while viewing a list or the like. The target device and control content may be specified.

In the embodiment, the control station 1
The functions for remotely controlling the video camera 4, the various functions such as the I / O map creation function, the I / O control operation function, and the like are processed by software, and the programs for each function are stored in a CDROM, a floppy disk, or the like. By storing the information in a recording medium, it can be widely provided. Then, by installing the program recorded on the recording medium into a personal computer, each function of the present embodiment can be realized. That is, the remote control device of the present embodiment does not require special hardware, and can be easily configured by a personal computer and a recording medium.

Each device, each unit, each operation, each process, and the like described in the above embodiments and modifications are implemented by means (up to means) as a general concept including them. It is possible to configure. For example,
The operation of receiving control target data from the I / O control device 6 can be configured by a control target receiving unit. Also, the camera frame 15 is displayed in the camera control window 152.
8, a camera frame display means, a camera frame moving means, and a camera frame size changing means may be used in correspondence with the description of the operation of moving and resizing the camera frame 158. Further, the I / O map data is stored in the storage device 26 or R
It is also possible to use an I / O map data storage unit corresponding to the operation to be stored in the AM 223 and an enlarged image acquisition unit corresponding to the operation of the enlarged image acquisition processing. Also, I /
The I / O control means corresponding to the O control device 6;
May be used as control means.

[0051]

According to the remote control device of the present invention, the control target can be easily recognized and specified, and the operability is improved.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of a remote operation system using a remote operation device according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of a personal computer in an operation side station.

FIG. 3 is an explanatory diagram of a main image window and a camera control window among display screens of the display device of the present embodiment.

FIG. 4 is an explanatory diagram of each divided screen when capturing all images.

FIG. 5 is a flowchart illustrating details of an I / O map creation process.

FIG. 6 is an explanatory diagram of an I / O map creation window displayed on a display device in an I / O map creation process.

FIG. 7 is an explanatory diagram of an I / O control window displayed on a display device in an I / O control operation.

FIG. 8 is a system configuration diagram of a conventional remote operation system.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Control side station 2 Operation side station 4 Video camera 5 Control part 6 I / O control device 7, 20 Mobile telephone 8 Modem 9 I / O control line terminal 11 Air conditioner 12 Television 13 Lighting 21 Personal computer 22 Control part 221 CPU 222 ROM 223 RAM 23 keyboard 24 mouse 25 display device 26 storage device 27 storage medium drive device 28 communication control device 29 input / output IF 151 main image window 152 camera control window 153 full screen capture key 154 detail setting key 155 setting stop key 156 setting transmission Key 157 Full screen 158 Camera frame 161 I / O map creation window 162 All image display section 163 Control target table display section 164 Mouse cursor 168 Operation panel 171 I / O control window

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hiroki Ishikawa 2--19-12 Sotokanda, Chiyoda-ku, Tokyo Inside Equos Research Inc. (72) Inventor Hiroshi Uehara 2-3-3 Tamagawadai, Setagaya-ku, Tokyo 20 Fifth YN Building 408 (72) Inventor Kiyoshi Fujinawa 2-6-1 1-101 Tachibanadai, Aoba-ku, Yokohama-shi, Kanagawa

Claims (3)

[Claims] 1. An image data obtaining means for obtaining image data, an image display means for displaying an image based on the image data obtained by the image data obtaining means, and an area on the image displayed by the image display means is designated. Area specifying means for performing the processing, correspondence means for associating the area specified by the area specifying means with a predetermined control signal, position specifying means for specifying a position on the image displayed on the image display means, Control signal output means for specifying an area specified by the area specifying means including the position specified by the specifying means, and outputting the control signal associated with the specified area by the corresponding means; A remote control device comprising: 2. An image data obtaining means for obtaining image data; an image display means for displaying an image based on the image data obtained by the image data obtaining means; and a region on the image displayed by the image display means. Area designating means, correspondence means for associating an area designated by the area designating means with a predetermined device to be controlled, position designating means for designating a position on an image displayed on the image display means, The area specified by the area specifying means including the position specified by the position specifying means is specified, and a plurality of control contents of the control target device associated with the specified area by the corresponding means are displayed. Control content display means for performing the control, control selection means for selecting one control content from the plurality of control contents displayed by the control content display means, Remote control device, characterized by comprising: a control signal output means for outputting a control signal corresponding to the control content, the. 3. The image acquisition unit acquires image data of an image of a device to be controlled, and the control signal output unit outputs the control signal to a control device that performs various controls on the device to be controlled. The remote control device according to claim 1 or 2, wherein: