JP2003291084A - Robot, its visual field control device and controlling method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a robot, its visual field control device and controlling method capable of properly controlling the visual field of a camera mounted on the robot and offering necessary communication information while the privacy is protected. <P>SOLUTION: The robot is fitted with the camera photographing images and equipped with a visual function capable of changing the direction of the line of sight, whereby the angle of the line of sight formed by the direction of the line of camera sight to a specified reference direction, for example the horizontal direction, is sensed for controlling the visual filed of the robot, and when the obtained angle of the line of sight is not less than a certain angle, the image given by the camera is subjected to an image processing such as dropping the resolution. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot having a visual function and acting in cooperation with a human, and a visual field control apparatus and method thereof.

[0002]

2. Description of the Related Art Who is a partner is indispensable information for a person who is engaged in social life. Therefore, it has been recognized that it is necessary to provide a visual function to machines such as robots that work in cooperation with humans. Among the visual functions, face image recognition technology has been rapidly advancing in recent years as an excellent personal identification function, and attempts have been made to apply it to machines such as robots.

As a conventional example of a robot having a general visual function, for example, there is an example of mounting a CCD camera or the like on a pet dog robot. This robot draws a human's attention by uttering a voice or performing a special action, and captures the image by capturing the human as a subject, and the captured image can be output to the outside. For example,
You can record captured images on a removable small memory card,
The communication function can be used to transfer a captured image to another device such as a mobile phone or a personal computer (PC).

The robot having such a visual function has the following problems unique to the robot.
In other words, this is a public order and moral problem that small pet robots are misused for so-called “voyeurism”. It is assumed that pet dogs and the like for pets are often brought into public places and the like, and they may be abused for voyeur photography by their owners, for example, while taking a walk in a park.

If a person clings to the ground and looks up, the woman will be wary because it is looking into something. However, when a pet robot such as a dog type does so, since the outer shape is a dog type, it is unlikely that a vigilant feeling is generated as it is for pet dogs of living things. Therefore, there is a possibility that improper use, such as approaching the feet of a woman and peeping under the skirt by the camera of the robot, will be made intentionally or by mistake. In this case, a particular problem is that the image taken by the robot is taken out.

[0006]

SUMMARY OF THE INVENTION The present invention has been made in consideration of such circumstances, and it is possible to appropriately control the field of view by a camera mounted on a robot and provide necessary communication information while protecting privacy. PROBLEM TO BE SOLVED: To provide a robot and a visual field control device and method thereof.

[0007]

In order to solve the above problems and achieve the object, the present invention is constructed as follows.

A robot according to the present invention has a camera for picking up an image, a detection means for detecting a line-of-sight angle formed by the line-of-sight direction of the camera and a predetermined reference direction, and a line-of-sight angle detected by the detection means is constant. And a field-of-view control means for controlling the field of view of the robot based on the image processing of the image captured by the camera at this time when the angle is greater than or equal to the angle.

Here, the visual field control means may be provided with an image processing means for reducing the resolution of the image picked up by the camera. In this case, the image processing for reducing the resolution of the image includes (1) reducing the overall resolution, or (2) extracting contour information of the subject and using this as an output image, or (3) information prepared in advance. Is output.

Further, it is provided with face image recognition means for recognizing a person's face from the image picked up by the camera, and the visual field control means for the face recognized by the face image recognition means and only the peripheral portion thereof. A configuration may be used in which the output image is extracted from the captured image.

Further, when the output means for outputting the image captured by the camera from the robot to the outside and the line-of-sight angle detected by the detection means are equal to or more than a certain angle, the image captured by the camera at this time Output control means for prohibiting output from the output means,
May be provided.

A view control device for a robot according to the present invention is
A visual field control device, comprising a camera for capturing an image, for controlling the visual field of a robot with a visual function capable of changing the visual line direction of the camera, wherein the visual line angle formed by the visual line direction of the camera and a predetermined reference direction is detected. And a visual field control means for controlling the visual field of the robot based on image processing of the image captured by the camera at this time when the line-of-sight angle detected by the detecting means is equal to or greater than a certain angle. And are provided.

Further, a robot visual field control method according to the present invention is a visual field control method which comprises a camera for picking up an image, and which controls the visual field of a robot with a visual function capable of changing the line-of-sight direction of the camera. A step of detecting a line-of-sight angle formed by the line-of-sight direction of the camera and a predetermined reference direction, and when the line-of-sight angle detected in the step is equal to or greater than a certain angle, the image captured by the camera at this time is image-processed. Based on this, a visual field control step of controlling the visual field of the robot is provided.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of a robot visual field control system according to an embodiment of the present invention, and FIG. 2 is a front view showing an example of a robot equipped with a visual function to which the robot visual field control system is applied ( It is a) and a side view (b). The robot shown in FIG. 2 is of a bipedal type, but is not limited to this, and various robots such as a quadrupedal type, a wheel-driven type, or a stationary type that does not have a moving ability can be used in the direction of the line of sight. The robot visual field control system according to the present invention can be applied as long as it has a visual function capable of moving.

The robot visual field control system according to the present embodiment is connected to the camera 2 and the sensor 6, and the image recognition unit 4, the image processing unit 8, the camera angle determination unit 10, the image compression unit 12, the determination unit 14, Output control unit 16, communication unit 17,
It is composed of an auxiliary storage device 19 and a small memory card 20. The robot according to the present embodiment uses an image of the outside world (for example, a subject such as a human) by the camera 2 having a visual function.
Can be captured as a still image or a moving image. The robot view control system according to the present embodiment appropriately controls the view of the camera 2 by image processing and provides necessary communication information while protecting privacy.

In the visual field control system of this embodiment,
The components that directly correspond to the present invention are the camera angle determination unit 10, the image processing unit 8, and the output control unit 16, and the other components can be appropriately omitted when implementing the present invention. For example, in the embodiment in which face image recognition is not performed, the image recognition unit 4
The determination unit 14 does not necessarily have to be provided.

The camera 2 and the sensor 6 are provided as a basic constituent part of the robot. The camera 2 is composed of, for example, a CCD camera, a lens optical system, and the like, and is arranged at a position of the face of the robot head 1 that allows a good view of the outside world as shown in FIG. As shown in the figure, this robot is roughly divided into a head 1, a main body 3, and legs 5.

The head 1 is connected to the main body 1 through a drive mechanism and can be tilted up and down or left and right, so that the head 1 moves up and down or turns right and left. . Further, the main body 3 is connected to the leg 5 via a drive mechanism and can be tilted forward or backward, so that the main body 3 makes a movement just like a person bending his or her waist. Further, the leg portion 5 has a drive mechanism portion at a position corresponding to a joint of a human knee or ankle, and makes a movement such that the human bends and extends the knee.

The sensor 6 shown in FIG. 1 comprises an angle sensor such as a rotary encoder provided in each movable part of such a robot, and is provided in a drive mechanism portion between the head 1 and the main body 3. This sensor 6 has a main body 3
60 for detecting an angle θ4 formed by the main axis of the head 1 with respect to the main axis (long axis) of the head, a sensor 61 for detecting an angle θ3 formed by the main body 3 with respect to the main axis of the thigh of the leg 5, and a leg 5
Angle θ2 formed by the thigh main shaft with respect to the shin main shaft
And a sensor 63 for detecting the angle θ1 formed by the vertical direction and the main axis of the shin of the leg 5. Signals corresponding to the detected angles are output from these sensors and sent to the camera angle determination unit 10 shown in FIG. A movable part is driven by each drive mechanism part, and the robot can walk or move autonomously or under external control. Further, it is possible to take various postures (positions) such as facing the front and looking up. Along with the posture change of the robot, the line-of-sight direction of the camera 2 also changes, and the field of view changes.

In the configuration shown in FIG. 1, the image recognition section 4
Is a part for recognizing a face image from a captured image obtained by the camera 2. Known techniques can be used for face image recognition. For example, the recognition is performed using one still image obtained from the camera 2 or using a plurality of images from different viewpoints. In the face image recognition processing using a multi-viewpoint image, first, a face area is extracted from the image obtained from the camera 2. Next, both pupils (pupils) and both nostrils are detected by the combination of the separability filter and pattern recognition. further,
A face pattern normalized based on these detected feature points is extracted. This series of processes is repeated while moving the viewpoint, and the input subspace is updated every time the normalized pattern is extracted. The input subspace and the dictionary subspace are projected onto the restricted subspace, and the canonical angle formed by the two projected subspaces is taken as the similarity. Such similarity calculation is performed for all registered persons, and the person having the highest similarity among the obtained similarities is recognized as the person.

The image recognition unit 4 as described above has a flag indicating whether or not a face image is included in the photographed image, position information indicating in which region of the photographed image the face portion exists, and the face image. The feature amount (face index) of is output.

In the configuration shown in FIG. 1, the image processing unit 8 performs angle condition determination processing on the line-of-sight angle (θcamera) of the camera 2 output from the camera angle determination unit 10, and obtains from the camera 2 according to the result. This is a part that performs image processing related to robot view control on the captured image. The image output from the image processing unit 8 is subjected to compression processing by the image compression unit 12, and then sent to the output control unit 16.
The contents of the image processing in the image processing unit 8 are matters relating to the features of the present invention, and will be described in detail later.

The determination unit 14 is a constituent element related to an application example to a so-called guard dog robot or a robot monitoring system in home use, and personal information based on the feature amount of the face image of the subject recognized by the image recognition unit 4. The person is identified by referring to the database. In the personal information database,
Face image recognition results (feature values) of these people are recorded as personal information of the family and acquaintances of the robot owners registered in advance. Such a determination unit 14 identifies a suspicious person or the like based on the reference result of the personal information database, transfers the captured image information (including the feature amount of the face image) to the outside, and outputs the alarm so as to issue an alarm. Give 16 instructions. In addition, as an example of application to so-called answering machine, when a child comes home from home, the answering robot takes a picture of the child with a camera and recognizes the face, and notifies the mother that the child is home to the fact that the child is home. , Transferring the child's face image, etc.

The output control unit 16 controls the output of the compressed captured image output from the image compression unit 12 to the communication unit 17, the auxiliary storage device 19, the small memory card 20, and the like. The output control unit 16 also uses the communication unit 17 and the auxiliary storage device so that a specific captured image is not output to the outside as a part of the robot view control based on the camera angle determination result, the image processing result by the image processing unit 8, and the like. It is also possible to perform control such that the output (writing) operation to the small memory card 20 or the like is prohibited.

The communication unit 17 is a communication function unit conforming to a wireless communication standard such as Bluetooth (R), IEEE 802.11, 802.11a, 802.11b (and later), and the output control unit 16 This is a part for transmitting a captured image or the like given by the wireless communication to an external device. The external device corresponds to a home server connected to the LAN, a home electric appliance, a security server, or the like. It should be noted that a wired communication function such as Ethernet (R) may be provided in addition to the wireless communication.

The auxiliary storage device 19 comprises a mass storage device such as a hard disk device. The small memory card 20 is a recording medium that can be attached to and detached from the robot via the slot.

With respect to the present embodiment configured as described above, the procedure of the robot visual field control according to the detection of the camera angle will be described with reference to the flowchart of FIG.

When the camera 2 of the robot 2 is activated and the photographing operation is started, first, at step S1, it is determined whether or not the angle of the camera 2 is a predetermined angle or more, that is, the camera 2
It is determined whether the line of sight of the person is abnormally upward. In addition,
The angle condition may be appropriately determined according to the outer dimensions of the robot, the arrangement position of the camera 2, the posture that the robot can take, and the like.

The detection of the line-of-sight angle (θcamera) of the camera 2 by the camera angle determination unit 10 is performed as follows.
FIG. 4 is a diagram showing a case where the robot of this embodiment shown in FIG. 2 is looking up. In this example, the ground plane of the robot is horizontal. Line-of-sight angle of camera 2 (θcamera)
Is an angle formed by the reference direction and the line-of-sight direction of the camera 2 when the horizontal is the reference direction, and the sensors 60, 61, 6
2 and 63 respectively output angle signals θ4,
By utilizing θ3, θ2, and θ1, it can be geometrically obtained as follows. θ camera = F (θ1, θ2, θ3, θ4) Normally, the robot is equipped with an inclinometer for motion control. This case is shown in FIG. In this example, the ground plane of the robot is a slope. Reference numeral 18 denotes an inclinometer, which is attached to the main body 3 having the largest mass in the robot.
The inclinometer 18 detects an angle θb formed by the main axis of the main body 3 and the vertical direction. Here, the line-of-sight angle of the camera 2 (θcame
Ra) can be calculated as follows using the angle signal θ4 output from the sensor 60 and θb output from the inclinometer 18. θcamera = F (θb, θ4) In addition, not only the inclinometer 18 but also a robot that expands the dynamic range by using a gyro sensor together is used. You may ask.

When it is determined in step S1 in FIG. 3 that the angle of the camera 2 is not equal to or larger than the predetermined angle, the image processing unit 8 sends the captured image of the subject obtained from the camera 2 to the image compression unit 12 as the original image. send. In this case, no image processing relating to the view control is performed on the captured image of the subject, and the image of the entire view is output. This is shown in FIG.
Shown in.

On the other hand, when it is determined in step S1 that the angle of the camera 2 is equal to or larger than the predetermined angle, that is, when it is determined that the line of sight of the camera 2 is abnormally upward as shown in FIG. 6B, It is determined whether or not the captured image obtained from the camera 2 includes a face portion (step S
3). This is determined based on the result of the face recognition process described above by the image recognition unit 4. In the case of the embodiment in which face recognition is not performed, the step is omitted and the process proceeds to step 5.

If it is determined in step S3 that the captured image includes a face portion, in step S4 image processing is performed to reduce the image resolution outside the periphery of the face. An example of such image processing is shown in FIG. In this figure, I0 indicates the original image of the image captured by the camera 2, and I1 indicates the processed image. Although the original image I0 is strictly different from the image of the entire field of view shown in FIG. 6B, it is shown in this way for convenience of explanation.
The image I1 is obtained by masking an image area outside the peripheral portion (here, an ellipse is a boundary) of the face of the subject with the mask image M as shown in the figure based on the recognition result of the face image by the image recognition unit 4. Is. Although this completely eliminates the resolution of the outer image area, the resolution may be reduced by performing mosaic processing or the like. The image processing in step S4 can be broadly called face image extraction processing. According to such extraction processing, it is possible to avoid a situation in which privacy is infringed due to a pirated image of the skirt of the subject or the like. Moreover, since the face image is always output regardless of the view control, the robot can provide necessary communication information about the subject when the captured image is externally transmitted in real time through the communication unit 17 or the like. become.

When it is determined in step S3 that the photographed image does not include a face portion, the following image processing relating to the visual field control is performed in step S5. That is, in this image processing, (1) the overall resolution of the captured image from the camera 2 is reduced, or (2) the contour information of the subject is extracted and used as the output image, or (3)
This is a process of outputting information prepared in advance. An example of these image processes is shown in FIG. The image I2 is obtained by superimposing the noise pattern N on the entire image region of the original image I0 of the captured image to reduce the resolution, and is the image processing example of (1) above. By reducing the resolution, the content of the captured image becomes difficult to see.

The image I3 is obtained by extracting the outer contour line O of the subject from the original image I0 and corresponds to the above (2). A known method can be used for the process of extracting the contour line from the image. Then, the image I4 cancels the original image I0,
Instead of this, the image of the message M such as "No image is output" is used. According to these image processes, when the line of sight of the camera 2 is abnormally upward as shown in FIG.
It is possible to avoid a situation where privacy is violated.
In a broad sense, the image processing of these step S5 blocks the visual field according to the angle detection result of the camera 2 in order to deal with the case where something to be concealed may exist in the visual field of the robot. This is equivalent to performing control based on image processing. Of course, the image processing for this purpose is not limited to that illustrated in FIG.

The robot view control according to the present invention is
It is not only based on the image processing as described above. For example, when it is recognized that the line-of-sight angle (θcamera) of the camera 2 output from the camera angle determination unit 10 is equal to or larger than a predetermined angle and the captured image includes a face image, the line-of-sight direction of the camera 2 is The operation of the drive mechanism section of the robot may be restricted so that the angle does not exceed that range. In this case, it is also effective to emit some warning sound or execute a special action sequence.

Further, the robot view control according to the present invention is
When a situation where something to be concealed may exist in the field of view of the robot is detected by detecting the line-of-sight angle of the camera and recognizing the face image, the image captured by the camera 2 at this time is prevented from being taken out of the robot. It also includes a control operation for performing. This is the output control unit 16 as described above.
Is a communication unit 17, an auxiliary storage device 19, a small memory card 2
This corresponds to the control for prohibiting the output (writing) operation of the captured image to 0 or the like.

As described above, according to the robot visual field control system of this embodiment, the visual field of the robot camera 2 can be appropriately controlled by image processing to provide necessary communication information while protecting privacy. it can.

The present invention is not limited to the above-described embodiment and can be variously modified and carried out.

[0040]

As described above, according to the present invention,
(EN) A robot capable of appropriately controlling a visual field by a camera mounted on a robot and providing necessary communication information while protecting privacy, and a visual field control device and method therefor can be provided.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a robot visual field control system according to an embodiment of the present invention.

FIG. 2 is a front view and a side view of the robot according to the same embodiment.

FIG. 3 is a flowchart showing an operation of the robot visual field control system according to the same embodiment.

FIG. 4 is a view showing a camera angle of the robot according to the embodiment, showing a case where the robot is looking up.

FIG. 5 is another view showing the camera angle of the robot according to the embodiment, showing a case where the robot is looking up on a slope.

FIG. 6 is a view for explaining a field of view and a subject of the robot according to the same embodiment.

FIG. 7 is a diagram for explaining an output image by the robot visual field control according to the embodiment.

FIG. 8 is a view for explaining another example of an output image by the robot visual field control according to the same embodiment.

[Explanation of symbols]

1 ... Robot head 2 ... camera 3 ... Robot body 4 ... Image recognition unit 5 ... Robot leg 6 ... Sensor 8 ... Image processing unit 10 ... Camera angle determination unit 12 ... Image compression unit 14 ... Judgment unit 16 ... Output control unit 17 ... Communication section 19 ... Auxiliary storage device 20 ... Small memory card

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Hideo Shimizu             1st Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa             Inside the Toshiba Research and Development Center (72) Inventor Tatsu Uebayashi             1st Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa             Inside the Toshiba Research and Development Center (72) Inventor Yasuhiko Suzuki             1st Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa             Inside the Toshiba Research and Development Center (72) Inventor Akio Okazaki             70 Yanagicho, Saiwai-ku, Kawasaki City, Kanagawa Prefecture             Toshiba Yanagimachi Office (72) Inventor Kenichi Hiramatsu             70 Yanagicho, Saiwai-ku, Kawasaki City, Kanagawa Prefecture             Toshiba Yanagimachi Office (72) Inventor Obama Wada             1-1 Shibaura, Minato-ku, Tokyo Co., Ltd.             Toshiba headquarters office (72) Inventor Koji Mikami             1-1 Shibaura, Minato-ku, Tokyo Co., Ltd.             Toshiba headquarters office (72) Inventor Akira Tojima             70 Yanagicho, Saiwai-ku, Kawasaki City, Kanagawa Prefecture             Toshiba Yanagimachi Office (72) Inventor Naoki Endo             No. 1 Toshiba-cho, Fuchu-shi, Tokyo Toshiba Corporation             Fuchu Office (72) Inventor Koichi Noguchi             1-1 Shibaura, Minato-ku, Tokyo Co., Ltd.             Toshiba headquarters office F-term (reference) 3C007 CS08 KS20 KS21 KS24 KT02                       MT00 WA03 WA13

Claims (12)

[Claims] 1. A camera for capturing an image, a detection means for detecting a line-of-sight angle formed by the line-of-sight direction of the camera and a predetermined reference direction, and a line-of-sight angle detected by the detection means being equal to or greater than a certain angle. And a visual field control means for controlling the visual field of the robot based on image processing of the image captured by the camera at this time. 2. The robot according to claim 1, wherein the field-of-view control means includes image processing means for reducing the resolution of the image captured by the camera. 3. A face image recognition means for recognizing a face of a person from an image captured by the camera, wherein the visual field control means includes the face recognized by the face image recognition means and only a peripheral portion thereof. The robot according to claim 1, wherein the robot is extracted from a captured image and used as an output image. 4. An output unit for outputting an image captured by the camera from the robot to the outside, and an image captured by the camera at this time when the line-of-sight angle detected by the detection unit is a certain angle or more. 4. The robot according to claim 1, further comprising an output control unit that prohibits the output from the output unit. 5. A visual field control device, comprising a camera for taking an image, for controlling the visual field of a robot with a visual function capable of changing the visual line direction of the camera, wherein the visual line direction of the camera and a predetermined reference direction are When the line-of-sight angle detected by the detecting unit and the line-of-sight angle detected by the detecting unit is a certain angle or more, the field of view of the robot is changed based on image processing of the image captured by the camera at this time. A visual field control device for a robot, comprising: a visual field control means for controlling. 6. The visual field control device according to claim 5, wherein the visual field control unit includes an image processing unit that reduces the resolution of an image captured by the camera. 7. A face image recognition means for recognizing a face of a person from an image captured by the camera is provided, and the visual field control means includes only the face recognized by the face image recognition means and a peripheral portion thereof. The visual field control device according to claim 6, wherein the visual field control device extracts the captured image as an output image. 8. An output unit for outputting an image captured by the camera from the robot to the outside, and a line-of-sight angle detected by the detection unit being a predetermined angle or more, the image captured by the camera at this time. The visual field control device according to any one of claims 5 to 7, further comprising: an output control unit that prohibits an image from being output from the output unit. 9. A visual field control method for controlling a visual field of a robot having a visual function capable of changing a visual line direction of the camera, wherein the visual line direction of the camera and a predetermined reference direction are different from each other. A step of detecting the line-of-sight angle to be formed, and when the line-of-sight angle detected in the step is equal to or greater than a certain angle, the field of view of the robot is controlled based on image processing of an image captured by the camera at this time. A field-of-view control method for a robot, comprising: a field-of-view control step. 10. The visual field control method according to claim 9, wherein the visual field control step includes an image processing step of reducing a resolution of an image captured by the camera. 11. A face image recognition step of recognizing a face of a person from an image captured by the camera, wherein the visual field control step captures only the face recognized in the face image recognition step and a peripheral portion thereof. 10. The image is extracted from an image and used as an output image.
Or the visual field control method according to any one of 10. 12. An output step of outputting an image captured by the camera from the robot to the outside, and when the line-of-sight angle detected in the detecting step is a certain angle or more, the image is captured by the camera at this time. 12. The visual field control method according to claim 9, further comprising: an output control step of prohibiting an image from being output in the output step.