JP4256850B2 - Information presenting apparatus, information presenting method, and information presenting program - Google Patents

Description

  The present invention relates to an information presentation technique for presenting accumulated information to one or more people.

  Conventionally, the following is used for information presentation in public spaces.

  Information presentation using posters, booklets, paper, and the like as a medium is inexpensive and is therefore very often used.

  A street terminal (large) has a large display and can provide information to many users at once.

  Considering services that offer information to users, such as advertisements, most services are aimed at getting many people to see, understand, and get interested. For this reason, it is effective to present information in a place where many people gather and get a lot of looks.

Moreover, in the street terminal, the system which can raise | generate interaction (information operation) from a user with respect to information is also disclosed (for example, refer nonpatent literature 1).
Kento Miyaoku, Shuguru Higasino, and Yoshinobu Tonomura, “C-Blink: A Hue-Difference-Based Il. 147-156, Oct. 2004

  In the conventional information presentation technique, a lot of information is presented at the same place. Therefore, there is a first problem that the information to be provided does not have a sense of unity and gives a miscellaneous impression to the user and impairs the scenery, giving an unpleasant feeling to a person who does not need the information.

  Moreover, since such a service is intended for many people, it is often a system that can be browsed by a plurality of people. In this case, the user browses standing information in front of the information he is interested in. However, the second problem is that there is a problem in terms of privacy because other people know what information they are viewing from the standing position. is there.

  In addition, as a third problem, when a plurality of people are viewing the same information, there is a problem that it is difficult for some users to stand in a direction facing the information, and it is difficult to see information to be viewed.

  In addition, in the information presentation apparatus, when the user wants to obtain other information, it is necessary to perform an information operation in order to reflect the user's intention on the information. Therefore, as a fourth problem, there is to provide an information presentation technique capable of performing an information operation.

  This invention is made | formed in view of this situation, The objective is to provide the information presentation technique which can solve the said subject.

Therefore, in order to solve the above problem, an information presentation apparatus according to claim 1 is an information presentation apparatus that provides information to a user using video presentation means, and waits for the user with respect to the video presentation means. The standby information presenting means for instructing to present information, the specific part detecting means for detecting the specific part of the user in the image from the image picked up by the image pickup means, and the image picked up using the image pickup means from the image, it detects the direction of the face of the user, and the user line-of-sight direction detecting means for determining the gaze direction of the user, and determine the orientation of presenting information from the viewing direction of the user, the detection on the image presentation means And detailed information presenting means for instructing to present the detailed information at the specified part .

The information presentation apparatus according to claim 2, wherein the information presentation apparatus includes user action recognition means for recognizing a certain action of the user, and the detailed information presentation means is configured to recognize the recognized action. It is characterized by performing the associated processing .

Further, the information presentation device according to claim 3 is the information presentation device according to claim 1 or 2, wherein the standby information presentation means indicates how close the user is to the area where the standby information is presented. Means for detecting whether the user is present, and the standby information is changed and presented in accordance with the degree of approach of the detected user .

The information presenting method according to claim 4 is an information presenting method in an information presenting apparatus that provides information to a user using a video presenting means, wherein the standby information presenting means is a user with respect to the video presenting means. A standby information presentation step for instructing to present standby information to the user, a specific part detecting unit for detecting a specific part of the user in the image from the image captured by the imaging unit, and a user A gaze direction detection unit detects a user's face direction from an image captured using the imaging unit, and a user gaze direction detection step for determining the gaze direction of the user, and a detailed information presentation unit include : determining an orientation for presenting information from the viewing direction, and instructs to present detailed information on the detected specific sites for the image presentation means It characterized by having a a fine information providing step.

Further, in the information presentation method according to claim 5 , the information presentation device includes user action recognition means for recognizing a certain action of the user, and the detailed information presentation means performs the recognized action on the recognized action. It is characterized by performing the associated processing .

The information presenting method according to claim 6 is the information presenting method according to claim 4 or claim 5, wherein the standby information presenting means is such that the user approaches a region where the standby information is presented. Means for detecting whether the user is present, and the standby information is changed and presented in accordance with the degree of approach of the detected user .

An information presentation program according to a seventh aspect is a program for causing a computer to function as each means constituting the information presentation apparatus according to any one of the first to third aspects.

In addition, in Claim 1 and 4, the image containing a user should just be an image which can detect a user's specific site | part.

According to the first and fourth aspects of the present invention, it is possible to provide information step by step, such as presenting standby information to the user and then presenting detailed information to a specific part of the user. Become.

And it becomes possible to provide information to a position where only the user who is the provision target of information can visually recognize the information.

Furthermore, since the information is provided based on the user's line of sight, the information can be provided in a manner that is easy for the user to see.

In the second and fifth aspects of the invention, information processing reflecting the user's intention can be performed by performing processing corresponding to the user's operation.

According to invention of Claim 1 and 4 , it becomes possible to solve the subject of giving a miscellaneous impression and giving a miscellaneous impression with respect to a user by providing information step by step, and a specific part. It is possible to solve the problem of giving unpleasant feeling to a person who does not need the information by presenting the information.

Then, by presenting the information at a position where only the user who is to provide the information can visually recognize the information, it is possible to solve the problem that other people know what information the user is viewing. .

Furthermore, since the information is provided based on the user's line of sight, when a plurality of people are viewing the same information, it is difficult for some users to stand facing the information, making it difficult to see the information that they want to view. Can be solved.

According to the second and fifth aspects of the present invention, when the user wants to obtain other information, the user performs an information operation that reflects the user's intention by performing a certain operation. It becomes possible.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram illustrating an information presentation apparatus according to the present embodiment. The information presentation apparatus in this embodiment includes at least a video presentation unit 1, an imaging unit 2, a specific part detection unit 3, a detailed information presentation unit 4, and a standby information presentation unit 5.

  The video presentation means 1 receives signals from the standby information presentation unit 5 that presents standby information and the detailed information presentation unit 4 that presents detailed information, and presents the corresponding video.

  The imaging unit 2 acquires a video and transmits the video to the specific part detection unit 3.

  The specific part detection unit 3 detects a part presenting detailed information from the transmitted video. Here, as the specific part, a position that can be easily seen by the user, such as a user's palm, the back of the hand, an arm, a toe, an item held by the hand, and an object worn by the user, or a position that can be seen only by the user can be considered. The information to be detected is the position information and size of the part, and the distance from the video presentation means 1 to the specific part. As the distance from the video presentation means 1 to the specific part, a value set in advance assuming an average human body shape may be used, or the distance sensor 10 shown in FIG. 2 may be used. You may ask for it. Information detected in this way is transmitted to the detailed information presentation unit 4.

  The detailed information presentation unit 4 receives information on the position and size of the specific part and the distance from the video presentation means 1 to the specific part from the specific part detection unit 3, and details to be presented according to the size of the specific part Reduce or expand the size of information. Furthermore, the focus of the image | video shown to a specific site | part is determined by the distance from the video presentation apparatus 1 to a specific site | part. Further, when the position of the specific part is within the video presentation area of the video presentation means 1, the detailed information presentation unit 4 instructs the video presentation means 1 to present the video so as to apply to the position. send.

  Here, the detailed information may be detailed information related to the standby information, or may be information on another matter not related to the standby information.

  Further, the video presentation means 1 for presenting standby information and detailed information is not limited to one, and a plurality of video presentation means 1 may be provided. The video presentation means for presenting standby information and the video presentation means for presenting detailed information may be different video presentation means.

  The video presentation means 1 for presenting standby information may be any means such as a projector, a liquid crystal display, and a plasma display that allows the user to view the video.

  In addition, an infrared camera or the like may be used as an imaging device that captures an image processed by the specific part detection unit 3 or the user motion recognition unit 6 and the user gaze direction detection unit 7 described later. In addition, the imaging device may include a plurality of imaging devices of the same type, or a plurality of imaging devices of different types.

  In the present embodiment, when the user is moving while browsing the information, the user gaze direction detecting unit 7 can be provided in order to continue presenting the information in a direction that is easy for the user to see. The user gaze direction detection unit 7 detects a human gaze direction from the video acquired by the imaging unit 2, transmits the detected gaze direction to the detailed information presentation unit 4, and presents detailed information from the gaze direction. The direction in which the unit 4 presents information is determined.

  Here, the detection of the line-of-sight direction will be described with reference to FIG. As shown in FIG. 2, a distance sensor 10 can be installed on the user's upper part in order to detect the line-of-sight direction. In this way, the distance sensor 10 is installed on the upper part of the user, and the user gaze direction detection unit 7 detects the head which is the highest part of the user with respect to the information from the distance sensor 10. The user gaze direction detection unit 7 determines that the direction from the center of gravity of the head region toward the center of gravity of a specific part that presents detailed information is the gaze direction.

  Note that the user gaze direction detection unit 7 may detect the direction of the user's face by using at least two or more imaging units 11 and 12 and extract the gaze direction. In this case, the imaging means 11 and 12 are placed at a position where the user's face can be imaged.

  In addition, when a user exists in the standby information presentation area of the information presentation device and the user is browsing detailed information, a user for extracting a user action from the image of the imaging unit 2 and operating the system The motion recognition unit 6 may be installed.

  The user action recognition unit 6 has an action database for managing a predetermined user action and an action ID corresponding to the predetermined user action, detects the user action from the picked-up video of the image pickup means 2, and then detects the detected user action. The corresponding operation ID is acquired from the operation database, and the acquired operation ID is sent to the detailed information presentation unit 4.

  The detailed information presentation unit 4 has an operation database for managing the operation ID and the information operation corresponding to the operation ID, and performs an information operation corresponding to the operation ID sent from the user operation recognition unit 6.

  Note that the user action may be anything using only the hand, bending the arm up and down, or movement of the entire body.

  Information operations include displaying information, updating information, rejecting information, changing the display size by scaling, or sending information to an address registered in advance by the user. Is mentioned.

  The structure of the information presentation apparatus used for a present Example is demonstrated using FIG. As shown in FIG. 3, the standby information presentation unit 5 is implemented in the computer 104 as software.

  The video presentation means 1 is implemented by a projector 101, and this projector 101 is connected to a computer 104.

  The imaging means 2 is implemented by a video camera 102, and this video camera 102 is connected to a computer 105.

  The specific part detection unit 3, the user gaze direction detection unit 7, the user motion recognition unit 6, and the detailed information presentation unit 4 are mounted in the computer 105, and the computer 105 and the computer 104 are connected by a network.

  Further, it is assumed that the projector 101 and the video camera 102 are attached to the ceiling of the building and are projected on the floor surface or photographed on the floor surface, respectively.

  Next, a specific operation in the present embodiment will be described. In the present embodiment, rain information is used as standby information, a palm is detected as a specific part of the user 103, and a line-of-sight direction is detected using the video camera 102.

  Here, the user action recognizes the hand action. Recognizing hand movements include a gripping movement, a shaking movement, a tilting movement, and a movement with both hands. Each of these hand actions is assigned an action ID.

  As information operations, operations of storing information, updating information, rejecting information, and increasing the amount of information are realized. Each of these information operations is associated with an action ID.

  When there is detailed information to be presented, the standby information presentation unit 5 sends the rain video generated by the computer 104 to the projector 101. The projector 101 projects the rain image sent from the standby information presentation unit 5 on the floor surface.

  In a rainy image, a person often performs an operation of receiving rain with a palm when it starts to rain. Therefore, in this embodiment, it is assumed that the user presents a hand by presenting a rain video. Then, the detailed information is presented on the handed out palm.

  In the present embodiment, the standby information presentation unit 5 may include means for detecting the user position. The means for detecting the user position is a means capable of detecting how close the user 103 is to the center of the area presenting the standby information. Thus, the standby information may be presented to the user so that the rain changes drastically as the user approaches the standby information.

  The video camera 102 captures the vicinity including the area where the standby information is presented, and the captured image of the video camera 102 is transmitted to the computer 105, and the specific part detection unit 3, the user gaze direction detection unit 7, and the user The motion recognition unit 6 obtains the transmitted captured image.

  Here, the process in the specific part detection part 3 is demonstrated using FIG.4 and FIG.5.

  The specific part detection unit 3 stores an image without a person as an initial image, and obtains a difference from the current image with a person (FIG. 4 (1)). As a result, the specific part detection means 3 obtains a difference image A (FIG. 4 (2)) of only the person region.

  The difference image A (FIG. 4 (2)) is a color image and has three regions of RGB. For this reason, it takes time to perform image processing with a large file size. Therefore, the difference image A (FIG. 4 (2)) is converted into a binary image of white and black, where the difference portion is white and the non-difference portion is black.

  Next, the white area of the binarized image is expanded and reduced. As a result, small noise in the image is removed, and a binarized image B (FIG. 4 (3)) is obtained.

  Next, the moment of the area of the binarized image B (FIG. 4 (3)) is obtained. The center of gravity 41 is detected from the obtained result, and this is used as the center of gravity of the person's body (FIG. 4 (4)).

  Only the skin color area C is extracted from the difference area of the difference image A (FIG. 4 (1)) (FIG. 4 (5)).

  An area having a size less than the predetermined minimum threshold in the area C and an area having a size exceeding the maximum threshold are removed, and from the position of the center of gravity 41 obtained previously in the removed area. A region D having the farthest center of gravity is obtained (FIG. 4 (6)).

  Here, there is a constraint that the palm region is located farthest from the center of gravity 41 of the body when viewing the video with the user's hand. From there, the region D becomes a region including a palm. However, this region is detected as a region including the arm region when the user is wearing short-sleeved clothing. For this reason, it is necessary to cut out only the hand region. This hand region is farther from the center of gravity of the body than the arm region. Therefore, a point 42 farthest from the center of gravity of the body is found in the region D. This point is included in the hand region, and can be said to be a point at the boundary with the background in the hand region. Therefore, a shared portion of the inner region of the circle 45 and the region D around this point is obtained. At this time, the radius of the circle 45 is set to the length from the middle finger to the wrist of a general human hand. By doing so, the area E which is a shared portion becomes the hand area. Further, the center of gravity 44 of the region E is obtained (FIG. 4 (7)).

  Next, it can be said that the boundary line between the region E and the circle 45 becomes a wrist region. Therefore, a point 43 serving as the center of gravity of the boundary line is obtained, and this point is assumed to be the center of gravity of the wrist (FIG. 4 (8)).

  Finally, it is detected that the center of gravity 44 of the obtained region E and the region 46 up to the center of gravity 43 are palms (FIG. 4 (9)).

  Information on the detected palm position and the size of the region 46 is transmitted to the detailed information presentation unit 4. Further, the specific part detection unit 3 obtains in advance the distance between the projector 101 and the palm that is the specific part from the average human body shape, and transmits this information to the detailed information presentation unit 4 as well.

  Further, the user gaze direction detection unit 7 detects a direction from the center of gravity 41 of the body transmitted from the specific part detection unit 3 to the center of gravity 44 of the hand as a gaze direction 47. This is because this direction is similar to the true line-of-sight direction. The user gaze direction detection unit 7 transmits the gaze direction 47 to the detailed information presentation unit 4.

  The detailed information presentation unit 4 receives information on the position and size of the palm that is the specific part and the distance from the projector 101 to the palm from the specific part detection unit 3, and applies the detailed information to the palm of the user from the information on the palm size. Scale up or down to size. Note that there are at least one or more detailed information to be presented, and they may be selected at random, or each may have an order and be presented in that order.

  Furthermore, the detailed information presentation unit 4 receives the user's line-of-sight direction from the user's line-of-sight direction detection unit 7 and rotates the detailed information so that the direction is the same as the user's line-of-sight direction. That is, when the user sees the detailed information, the characters are not reversed. Further, the detailed information presentation unit 4 sends a command to the projector 101 to present the rotated detailed information to the palm position sent from the specific part detection unit 3. Further, the detailed information presentation unit 4 determines the focus of the projector 101 based on the distance from the projector 101 to the palm sent from the specific part detection unit 3.

  Based on such information, the projector 101 presents detailed information to the user's palm. At this time, if the area where the detailed information is presented is outside the video presentation range of the projector 101, the detailed information is not presented, the number of projectors is increased, and the entire imaging range of the video camera 102 can be covered. Alternatively, it is possible to handle the projector so that the detailed information is presented using a device driven by the projector.

  The user motion recognition unit 6 obtains the user's hand center of gravity 44, the body center of gravity, and the size of the hand region (region E) by the same processing as the specific part detection unit 3.

  Here, an operation database table of user operations and operation IDs will be described with reference to FIGS.

  Below, an example of the procedure in which the user action recognition part 6 recognizes a user action is demonstrated.

  The user motion recognition unit 6 recognizes that it is a gripping motion when the size of the hand region is less than two-thirds of the current size and the center of gravity of the hand approaches the center of gravity of the body.

  Further, the user motion recognition unit 6 recognizes that the motion is tilted when the size of the hand region is less than two-thirds of the current size and the position of the center of gravity of the hand does not substantially change.

  In addition, when the size of the hand region is double the current size, the user action recognition unit 6 recognizes that the action is to add both hands.

  As shown in FIG. 6, recognized user actions are assigned action IDs by the action database. When the user action is recognized, the user action recognition unit 6 transmits this action ID to the detailed information presentation unit 4. Further, information on the hand region detected by the above-described processing is also transmitted to the detailed information presentation unit 4.

  The detailed information presentation unit 4 receives the operation ID from the user operation recognition unit 6, and performs an information operation corresponding to the operation ID using the information operation database shown in FIG. The detailed information presentation unit 4 also receives hand region information from the user action recognition unit 6 and uses the information when hand region information is necessary for performing information operations.

  Here, an example of the above-described process will be described. The detailed information presentation unit 4 makes an inquiry to the information operation database when receiving the operation ID 04 that is an operation with both hands. Based on the information operation returned from the information operation database, the detailed information presentation unit 4 performs a process of increasing the information amount on the presented detailed information. Here, the process of increasing the amount of information is enlargement or detailing of detailed information.

  In addition, when the user performs an operation of adding both hands, the hand area is enlarged. As a result, the detailed information presentation unit 4 sends a command to the projector 101 to enlarge the detailed information to fit the enlarged hand region transmitted from the user action recognition unit 6 or to present more detailed detailed information. .

  The projector 101 projects the information sent from the detailed information presentation unit 4 onto the user's palm so as to meet the command.

  The information presentation device is realized by, for example, a CPU included in a computer device that constitutes the information presentation device, and requires a specific part detection function, a detailed information presentation function, a standby information presentation function, a user action recognition function, a user line of sight. A direction detection function or the like can be installed as an application program.

  In addition, a recording medium in which a program code of software that realizes the above functions is recorded is supplied to a system or apparatus, and a CPU (MPU) of the system or apparatus reads and executes the program code stored in the recording medium. realizable. In this case, the program code itself read from the storage medium realizes the above function. Examples of the storage medium storing the program code include a CD-ROM, a DVD-ROM, a CD-R, a CD-RW, an MO, and an HDD.

The block diagram of an information presentation apparatus. The block diagram of an information presentation apparatus. Explanatory drawing of the condition which provides information to a user using an information presentation apparatus. Processing explanatory drawing in a specific part detection part. The partially expanded view of the process description in a specific part detection part. The block diagram of an operation | movement database. The figure which shows the example of operation | movement. The block diagram of an information operation database.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image | video presentation means 2 ... Imaging means 3 ... Specific site | part detection part 4 ... Detailed information presentation part 5 ... Standby information presentation part 6 ... User action recognition part 7 ... User gaze direction detection part 9 Infrared camera 10 ... Distance sensor 11 ... Imaging Means 12 ... Imaging means 101 ... Projector 102 ... Video camera 103 ... User 104 ... Computer 105 ... Computer

Claims (7)

An information presentation device that provides information to a user using video presentation means,
Standby information presentation means for instructing the video presentation means to present standby information to the user;
Specific part detecting means for detecting a specific part of the user in the image from the image picked up by the image pickup means ;
User gaze direction detection means for detecting the orientation of the user's face from the image captured using the imaging means and determining the user's gaze direction;
Detailed information presenting means for determining a direction of presenting information from the user's line-of-sight direction and instructing the video presenting means to present detailed information to the detected specific part ;
An information presentation device characterized by comprising: The information presenting apparatus has user action recognition means for recognizing a certain action of the user,
The information presentation apparatus according to claim 1 , wherein the detailed information presentation unit performs processing associated with the recognized action.
The standby information presenting means includes means for detecting how close the user is to the area where the standby information is presented, and changes the standby information according to the degree of approach of the detected user. Present
The information presentation apparatus according to claim 1, wherein the information presentation apparatus is an information presentation apparatus. An information presentation method in an information presentation device for providing information to a user using a video presentation means,
A standby information presenting step in which the standby information presenting means instructs the video presenting means to present standby information to the user ;
A specific part detecting step for detecting a specific part of the user in the image from the image picked up by the image pickup means ;
A user gaze direction detecting unit that detects a user's face direction from an image captured using the imaging unit and determines the user's gaze direction;
A detailed information presenting step in which detailed information presenting means determines a direction in which to present information from the user's line-of-sight direction, and instructs the video presenting means to present detailed information to the detected specific part ; ,
An information presentation method characterized by comprising: The information presenting apparatus has user action recognition means for recognizing a certain action of the user,
The information presentation method according to claim 4 , wherein the detailed information presentation unit performs a process associated with the recognized action. The standby information presenting means includes means for detecting how close the user is to the area where the standby information is presented, and changes the standby information according to the degree of approach of the detected user. Present
The information presentation method according to claim 4 or 5, characterized in that   The information presentation program for functioning a computer as each means which comprises the information presentation apparatus of any one of Claim 1 to 3.

Images