JP2005056213A - System, server and method for providing information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for properly changing the contents or level of information to be provided when the system is applied to various spaces such as a home, a day nursery, a nursing-care facility, a hospital, an office, a hotel and a store, so as to provide the information concerning the situations in the spaces. <P>SOLUTION: The system includes a sensing means 120 for picking-up the inner part of the space; an article/mobile body retrieval management means 105 for specifying a situation in the space; an image management means 104 for recognizing information concerning a mobile body and/or an article, which are included in the picked-up image data; a display control means 118 for processing the image data through the use of a recognition result in response to processing contents corresponding to the specified situation by referring to a display control database 123 for storing the situation in the space and the processing contents of the image data with association; and a display part 117 for displaying the processed image data. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an information providing system, an information providing server, and an information providing method for providing information related to a situation in a predetermined space.

  For example, Patent Document 1 discloses a system that provides image data obtained by capturing the state of a child as information relating to a situation in a predetermined space (daycare).

This information providing (monitoring) system has an imaging control means for controlling a camera for imaging a child in the daycare center, and the imaging control means is a camera for imaging the child of the user in response to a request from the user. The system is configured such that an angle is determined, a zoomed-in image is captured, and the image is distributed to users via a network. With this configuration, since the delivered image includes only my child and does not include other children, the user checks the state of my child in the daycare center while protecting the privacy of each other's children. be able to.
JP 2002-269209 A

  The information providing system disclosed in Patent Document 1 is a system that images a specific person in a space by controlling camera angle and zoom-up for the purpose of protecting privacy. That is, since the information providing system disclosed in Patent Document 1 is a system that captures image data in which the included information is limited to one, for example, while the child of another person is not included in the image, the entire daycare center It is impossible to include both the state and the state of my child in the image.

  The present invention has been made in view of such points, and the object of the present invention is to be applied to various spaces such as general households, daycare, nursing care facilities, hospitals, offices, hotels and stores, and the like. It is an information providing system that provides information related to the situation in the space, and is to realize a system that can appropriately change the content and level of the information to be provided.

  The present invention processes image data obtained by imaging a predetermined space in accordance with the situation in the space.

  The information providing system of the present invention is a system that provides information about a situation in a predetermined space.

  The information providing system includes an imaging unit that images the inside of the space, a specifying unit that specifies a situation in the space captured by the imaging unit, and a moving body and / or article included in image data captured by the imaging unit. The identifying means identified by referring to the recognizing means for recognizing the information, the processing content management database storing the situation in the space and the processing content of the image data in association with each other, and the processing content management database Processing means for processing image data captured by the imaging means using the recognition result of the recognition means in accordance with the processing content corresponding to the situation, and display means for displaying the processed image data.

  Here, the moving body includes a person, an animal, a robot, and the like, and performs various activities in the space.

  According to this configuration, when the imaging unit images the inside of the space, the specifying unit specifies the situation in the space, and the recognition unit recognizes information about the moving object and / or the article included in the image data.

  Then, the processing unit refers to the processing content management database and processes the image data captured by the imaging unit using the recognition result of the recognition unit according to the processing content corresponding to the situation specified by the specifying unit. . In this way, the image data is processed according to the situation in the space, whereby it is possible to perform processing such as deleting necessary information from the image data while including necessary information in the image data.

  The processed image data is displayed by the display means. The user can check the state in the space from the displayed image data.

  The information providing system further includes operation means for outputting a display request signal for image data in response to a user operation, and the processing content management database corresponds to the user information and the situation in the space and the processing content of the image data. The processing means receives the display request signal, acquires the user information related to the output of the display request signal, and according to the processing content corresponding to the user information and the situation specified by the specifying means Image data may be processed.

  The processing content management database may store a combination of an attribute of the space and an attribute of a moving object and / or an article existing in the space as a situation in the space.

  The information providing system further includes sensing means for detecting a moving object and / or article existing in the space, and the specifying means specifies a situation in the space based on a detection result of the sensing means, and The recognizing unit may recognize information about the moving object and / or the article included in the image data based on the image data captured by the imaging unit and / or the detection result of the sensing unit.

  The recognition means recognizes information about the moving object and / or article for each of a plurality of information levels, and the processing content management database recognizes the situation in the space and the recognition result of the information level selected according to the situation. May be stored in association with the processing content set to be included in the processed image data.

  Here, the information level is obtained by individually dividing a large amount of information included in image data. For example, when a moving object is included in the image data, 1) presence / absence of the moving object, 2) what (who) the moving object (attribute of the moving object), 3) position of the moving object, 4) moving object Examples of information levels include the position of the face, 5) the contour shape of the moving body, 6) what the moving body is doing (movement of the moving body), and the like.

  The information providing system further includes an image management database that stores the image data captured by the imaging unit and the recognition result recognized by the recognition unit in association with each other, and the processing unit is stored in the image management database. The image data may be processed using the recognition result stored in the image management database.

  The information providing server of the present invention is a server that provides information related to a situation in a predetermined space.

  The information providing server includes a specifying unit for specifying a situation in a space captured by the imaging unit, a recognition unit for recognizing information about a moving object and / or an article included in the image data captured by the imaging unit, By referring to the processing content management database that stores the situation in the space and the processing content of the image data in association with each other, according to the processing content corresponding to the situation specified by the specifying unit, using the recognition result of the recognition unit Processing means for processing image data picked up by the image pickup means, and output means for outputting the processed image data.

  The information providing method of the present invention is a method for providing information related to a situation in a predetermined space.

  The information providing method includes an imaging step of imaging the space, a specifying step of specifying a situation in the space imaged in the imaging step, and a moving object and / or article included in the image data captured in the imaging step A recognition step for recognizing information about the image, and a processing step for processing the image data captured in the imaging step using the recognition result in the recognition step according to the processing content set in correspondence with the situation specified in the specifying step And a display step for displaying the image data processed in the processing step.

  According to the information providing system, the information providing server, and the information providing method of the present invention, it is possible to change the content and level of information to be provided by processing captured image data according to the situation in the space, thereby making it necessary. It is possible to realize a flexible response such that unnecessary information is included in the image data and unnecessary information is deleted from the image data.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. An information providing system (monitoring system) according to the present invention is a system included in an article / moving body management system that manages articles and people (moving bodies) in a predetermined space. This monitoring system monitors the space using an imaging means (sensing means) used in the article / moving body management system. Note that the predetermined space is a space (a living space) in which a person performs an activity such as a general home, a daycare, a nursing facility, an office, a hotel, a store, and a hospital. In the present embodiment, the target space is a general home. And In addition, each room in a general home is called a space, and each space (room) is called an environment.

  As shown in FIG. 1, the article / moving body management system includes an environment management server 101 (hereinafter simply abbreviated as a server), a work robot 102 (hereinafter abbreviated as a robot), and The operation terminal 103 is roughly divided into three subsystems. These three subsystems 101 to 103 are connected via a wireless or wired network and configured to exchange information via this network.

  Each of the three subsystems 101 to 103 includes control means 110, 115, 119 and transmission / reception means 109. Since the transmission / reception means 109 of each of the subsystems 101 to 103 has the same processing, the same reference numerals are given.

  Here, the article refers to an item handled by a person or work robot 102. The person and the work robot 102 handle articles, and are hereinafter collectively referred to as a moving body. Furthermore, a mobile object that handles an article is called an article handling entity.

-Configuration of sensing means-
The article / moving body management system includes sensing means 120 for grasping the situation in each environment, and the sensing means 120 is connected to the environment management server 101 which is the first subsystem.

  The sensing means 120 includes an entrance / exit sensing means 121 for detecting the state of the entrance / exit of each environment and an in-environment sensing means 122 for detecting the situation in each environment.

(Entrance / entrance sensing means)
The entrance / exit sensing means 121 detects that an article and a moving body have entered and exited the environment. The entrance / exit sensing means 121 is installed for each environment.

  As the entrance / exit sensing means 121, a device comprising an electronic tag and a reader / writer can be employed.

  An electronic tag is a device composed of an IC that stores data and an antenna that transmits and receives data wirelessly. A reader / writer reads information written to the electronic tag and writes information to the electronic tag. It is a device that can.

  For example, as shown in FIG. 2, the entrance / exit sensing means 121 composed of the electronic tag and the reader / writer has a gate type reader / writer (RF antenna) at each of the openings of the window 51 and the door 52 which are the entrance and exit of the environment and the outside. ) 41 and 42 are provided.

  In addition, the electronic tag is attached in advance to all articles that pass through the doorway. The electronic tag may be embedded with data relating to the article, such as data such as the kind, shape, weight, article image, date of manufacture, etc. of the article.

  Further, an electronic tag is also attached in advance to the moving body that passes through the doorway. An electronic tag attached to a person may be embedded in something that the person always carries (for example, a wristwatch or a ring). It is preferable to write data relating to the moving body, for example, information such as a person's name and date of birth on the electronic tag attached to the moving body.

  By configuring the entrance / exit sensing means 121 in this way, when the article or the moving body passes through the window 51 or the door 52, the reader / writers 41 and 42 installed at the entrance / exit are attached to the article or the moving body. Information from the electronic tag is read. Thus, by reading information from the electronic tag, the entrance / exit sensing means 121 can detect that the article and the moving body have passed through the entrance / exit.

  Further, if the reader / writer is installed twice with respect to the entrance / exit, the entrance / exit sensing means 121 can detect whether the article and the moving body have entered the environment or the exit from the environment.

  That is, although illustration is omitted, the first reader / writer is installed outside the opening of the window 51 and the door 52 outside the environment, while the second reader / writer is inside the environment, the window 51 and the door 52. It is installed inside the opening. With this configuration, when the first reader / writer installed outside the environment detects information from the electronic tag and the second reader / writer installed inside detects the information from the electronic tag, the electronic tag The attached article or moving body can be identified as having entered the environment from the outside. Conversely, when the second reader / writer installed inside the environment detects information from the electronic tag and the first reader / writer installed outside detects information from the electronic tag, the electronic tag is attached. The taken article or moving body can be identified as having gone out of the environment.

  In FIG. 2, the reader / writers 41 and 42 are installed so as to surround the upper, lower, left and right sides of the openings of the window 51 and the door 52. However, this is because detection is performed with high accuracy without depending on the direction of the electronic tag. The reader / writers 41 and 42 may be installed only at the vertical position of the window 51 and the door 52, only at the horizontal position, or at the center position thereof.

  Note that the entrance / exit sensing means 121 is not limited to the one composed of the electronic tag and the reader / writer, and may have other configurations.

(Environmental sensing means)
The in-environment sensing means 122 constantly detects the articles and equipment (including furniture etc.) existing in the environment, and the positions and states of people and the robot 102, and is installed for each environment. The in-environment sensing means 122 is configured here using an image sensor. That is, the in-environment sensing means 122 includes a camera that is fixed to the ceiling or wall of the room and captures the state of the environment. Detection is performed.

  Here, a method for detecting an article and a moving body will be described. One common method for detecting articles and moving objects in the environment using camera images is the background difference method, and the background difference method can be used as the environment sensing means 122. The background difference method is a method of detecting a target object by preparing a model image as a background in advance and taking a difference between the current camera image and the model image. Since the environmental sensing means 122 detects articles and moving objects present in the environment, the model image may be displayed when there are no articles or moving objects in the environment if there is little environmental fluctuation. What is necessary is just to use the imaged image. In the case where the environmental situation changes greatly, an image obtained by averaging a plurality of images taken with a predetermined time interval may be used as a model image.

  Specifically, an article and moving object detection method based on the background subtraction method will be described with reference to FIG. Here, FIG. 3A is a diagram showing an example of a model image, FIG. 3B is a diagram showing an image (input image) taken by the camera at a certain point, and FIG. It is a figure which shows the example of the background difference image obtained by deducting a model image from an input image. As can be seen from FIG. 3C, in the background difference image, a portion having a difference between the input image and the model image appears (shaded area reference (difference area) in FIG. 3). For this reason, the articles | goods or moving body which exists in an environment can be detected by taking out only the part which protruded, respectively. Further, the image is processed to identify what the article or moving body is (this processing is performed by the article / moving body search / management means 105 described later). As described above, by using the background subtraction method, it is possible to detect the status of articles and moving objects in the environment. Furthermore, if two cameras are provided in the environment, it becomes possible to acquire information on the shape and orientation of the article using a three-dimensional measurement technique based on stereo vision.

  If the above-described electronic tag and reader / writer are used, the article and the moving body can be easily identified by the information embedded in the electronic tag. Therefore, the environmental sensing means 122 uses both the image sensor and the electronic tag. It is also possible. That is, a hybrid process is performed in which the position of the article and the moving object in the environment is specified by the background difference method described above, and the article and the moving object are specified using the electronic tag. The following two examples can be given as specific processing.

  In one example, a camera is installed on a ceiling or wall in the environment, and a reader / writer is attached to the work robot 102 to constitute the in-environment sensing means 122. Moreover, an electronic tag is attached to each article and the moving body. Then, the position of the article or moving body in the environment is specified by the background subtraction method using the camera image. Next, the robot 102 is moved in the vicinity of the positioned article or moving body, and the reader / writer attached to the robot 102 reads information from the electronic tag attached to the article or moving body, and the article or Identify moving objects.

  In another example, the in-environment sensing means 122 is configured by installing cameras on a ceiling or a wall in the environment and installing a plurality of reader / writers in the environment substantially evenly. The reader / writer is assumed to have directivity with respect to data reading of the electronic tag and the reading direction is variable. Then, the position of the article or moving body in the environment is specified by the background subtraction method using the camera image. Next, a reader / writer installed at a position closest to the specified article or moving body is selected, and the reading direction of the reader / writer is directed to the article or moving body. In this way, information is read from the electronic tag attached to the article or moving body, and the article or moving body is specified.

  When the sensing means 120 detects an article or a moving body as described above, the sensing result is transmitted to the article / moving body search / management means 105 of the server 101.

-Configuration of environmental management server-
The environment management server 101, as shown in FIG. 1, out of the situation grasped by each of the entrance / exit sensing means 121 and the in-environment sensing means 122, the article / movement that manages the situation between the article and the moving body existing in the environment. Body search / management means 105, article / moving body database 160 for storing data of the article and moving body, environment map management means 107 for managing the situation of the entire environment other than the article and moving body, and data of the entire environment The environment map 108 for storing the image, the image management database 106 for storing the image data taken by the camera, the image management means 104 for managing the data stored in the image management database 106, the data of the article / mobile object database 160, Receives data queries (signals) from the environment map 108, sends response signals to the outside, and robot 1 A transmission / reception means 109 for transmitting a control command for 02 and a control means 110 for controlling these means 104, 105, 107, 109 are comprised.

  As the environment management server 101, a general-purpose computer can be used. In this case, the computer can be used as the environment management server 101 when the computer reads a control program for executing each process described later.

(Goods / mobile object search / management means)
The article / moving body search / management means 105 accumulates information on each article and moving body detected by the sensing means 120 in an article / moving body database (DB) 160.

  The article / moving body search / management means 105 identifies the article and the moving body based on the detection result of the sensing means 120, and accumulates the identification results in the article / moving body DB 160. Thus, the situation in the environment is specified by specifying the article and the moving body.

  Further, when there is an inquiry from the control means 110 to the article / mobile object DB 160, the article / mobile object search / management means 105 takes out necessary information from the article / mobile object DB 160 according to the content of the inquiry, The information is also sent to the control means 110.

  As shown in FIG. 4, the article / moving body search / management means 105 receives the detection result of the sensing means 120, and identifies an article specifying means 32 for specifying an article and a moving body specifying means 33 for specifying a moving body. .

  The article specifying means 32 specifies what the article is detected by the sensing means 120, and stores information on the article in the article / mobile DB 160. As described above, the article specifying unit 32 specifies an article based on information from the electronic tag when the sensing unit 120 uses an electronic tag and a reader / writer. Further, when the sensing means 120 uses a camera image, an article recognition process is performed based on the camera image and the article is specified.

  The moving body specifying means 33 specifies who is the moving body detected by the sensing means 120 and accumulates information on the moving body in the article / moving body DB 160.

  As described above, when the sensing unit 120 uses an electronic tag and a reader / writer, the mobile unit specifying unit 33 specifies the mobile unit based on information from the electronic tag.

  When the sensing means 120 uses a camera image, the mobile body is authenticated based on the camera image and the mobile body is specified. Specifically, the camera captures an area where the moving body is present, performs face authentication processing on the captured image, and identifies the moving body by the authentication processing. Here, since the camera used as the in-environment sensing means 122 is a camera that captures the entire environment, it is usually a wide-angle camera that captures a wide area. The captured image of this wide-angle camera has a relatively low resolution, and there are cases where the resolution is insufficient to perform face authentication processing. Therefore, apart from the camera used for the background subtraction method, a narrow-angle high-resolution camera may be installed in the environment or the robot 102 as a camera for face authentication processing. In this case, the moving body specifying unit 33 causes the narrow-angle camera to capture an area where the moving body exists, and performs face authentication processing on the captured image. By doing so, the moving body specifying means 33 can specify the article handling entity with high accuracy. The moving body specifying method in the moving body specifying means 33 is not limited to the face authentication process, and may be performed by other authentication processes such as iris authentication.

  Further, the article / moving body search / management means 105 detects that the article is handled by the moving body, and in this case, identifies the article and the moving body (article handling entity) that handles the article.

(Article / Moving Object Database)
The article / moving body DB 160 of the environment management server 101 is a DB for storing information on the article and the moving body, and is configured as shown in FIGS. 5 and 6, for example. That is, the article / moving body DB 160 includes an article database 61 (FIG. 5) that handles articles and a moving body database 62 (FIG. 6) that handles moving bodies. As will be described later, the information stored in the article / moving body DB 160 is also used when specifying the situation in the environment.

  The article DB 61 is composed of sub-databases each storing three types of data, article data 611, article history data 612, and article attribute data 613. The data contents stored in each sub-database are as follows.

  1) Article data 611: An ID for distinguishing individual articles, a pointer to article history data, and a pointer to article attribute data describing the attributes of the article are stored. Here, if a plurality of articles are of the same type but are physically different, different IDs are assigned to treat them as different articles, and pointers to different article attribute data are set. Have. On the other hand, since articles of the same type have the same article attributes, they have pointers to the same article attribute data even if different IDs are assigned. This saves database capacity.

  2) Article history data 612: Stores the history of handling articles in the environment, and consists of four items: handling time (handling date and time), handling contents, handling subject, and position after handling. Among these, the handling contents are expressed as new, movement, and the like. In addition, various representations of the position data can be considered. Here, the position is represented by six parameters as shown in FIG. That is, the first three parameters (x1, y1, z1) represent the position of the article (using the center of gravity position or the like), and the latter three parameters (α1, β1, γ1) represent the direction of the article. The position and orientation of the article are acquired by the sensing means 120 or the sensor 111 of the work robot 102. Further, the article handling entity in the article history data is a moving body specified by the moving body specifying means 33. In the article history data, the latest data in the “position after handling” is information on the current position of the article.

  3) Article attribute data 613: This stores physical attribute information possessed by an article. Examples of attribute information include the weight of the article and the shape model size as shown in FIG. Such attribute information may be acquired from an electronic tag attached to the article and stored in a database, or information manually input by a user may be stored in the database.

  As shown in FIG. 6, the mobile object DB 62 is composed of sub-databases each storing three types of data: mobile object data 621, mobile object history data 622, and mobile object attribute data (not shown). The contents of the accumulated data are as follows.

  1) Mobile object data 621: An ID for distinguishing individual mobile objects and a pointer to mobile object history data are stored. The mobile body stored in the mobile body data 621 may be registered in advance manually by the user.

  2) Moving object history data 622: It consists of three items: the position of the moving object at a certain time and the state at that time. Here, unlike the article, the moving body occupies a large volume in the space and becomes an obstacle when the robot 102 moves in the environment. For this reason, it is preferable that the position of the moving body be expressed as realistic as possible. Here, an approximate area occupied by the moving object on the floor surface is represented by a circle. That is, the position of the moving object is represented by the XY coordinates (x4, y4) of the center position of the circle and the radius (r1) of the circle. This is because the robot 102 can create a route while avoiding an obstacle, while expressing the position of the moving body with the minimum necessary information. Note that the region occupied by the moving object may be expressed more strictly. For example, the position of the moving body may be expressed using a plurality of line segment vectors that approximate the outline of the area occupied on the floor of the moving body.

  In addition, the state of the moving object in the moving object history data represents general human actions such as “sitting”, “standing”, “sleeping”, and “walking” if the moving object is a person. 102 represents an operation performed by the robot 102 on the article such as “gripping” and “releasing”. For example, a plurality of state candidates that can be in the state of the moving body are prepared in advance, and it is only necessary to determine to which state candidate the state of the moving body applies based on the detection result by the sensing means 120 or the like. In the moving body history data related to the robot 102, not only the operation but also the article ID of the work target is stored as “article ID: operation content”. Specifically, “kan-small-0001: grip”.

  3) Mobile object attribute data: Accumulated shared attribute information possessed by the mobile object. Examples of the mobile object attribute data include physical attributes such as the weight and shape of the mobile object. This mobile attribute data can be acquired by the sensing means 120 (electronic tag and reader / writer). Moreover, you may input such information manually.

  Access to the article / mobile body DB 160 is permitted only to the article / mobile body search / management means 105, thereby preventing writing and reading of the same article or mobile body data from being performed simultaneously.

  Next, how the article detected by the sensing means 120 and information on the moving body such as the person and the robot 102 is stored in the article / moving body DB 160 by the article / moving body search / management means 105, Further, how the data of the article / moving body DB 160 is updated will be specifically described with reference to FIGS.

  FIG. 7 is an overhead view showing the inside of a room (environment) in a general home. Two can juices 21 and 22 enter and leave the room, and these can juices 21 and 22 are in the environment. It is a figure which shows a mode that is moved.

  In the figure, the movement of each can juice 21 and 22 is indicated by arrows, and the letters t1, t2, t5, and t6 attached to these arrows indicate the time when the can juice was moved according to the arrow. Indicates. It is assumed that the time has advanced in the order of t1 to t6.

  Although illustration is omitted at the entrance / exit (door) of the room, a reader / writer is installed as the entrance / exit sensing means 121 to detect the entry and exit of articles and moving objects separately, and in the room It is assumed that a camera and a reader / writer are installed as the in-environment sensing means 122 to detect the movement of articles and moving objects in the environment.

  In addition, it is assumed that each of the can juices 21 and 22 and the person entering and exiting the room and the robot 102 are each provided with an electronic tag. The article attribute data of each of the can juices 21 and 22 is obtained by reading information embedded in the electronic tag by a reader / writer. It is assumed that information on each moving body is registered in the moving body data 621 of the moving body DB 62, and the moving body history data 622 is already initialized. Further, it is assumed that the article data 611 of the article DB 61 is an empty situation in which nothing is recorded.

  First, a father (not shown) registered in the moving body data 621 in the moving body DB 62 enters the room from the door with the can juice 21. At this time, the entrance / exit sensing means 121 provided in the door detects that the father and the can juice 21 have moved into the room.

  The detection result is sent from the sensing means 120 to the article / mobile body search / management means 105, and the article / mobile body search / management means 105 receiving the detection result identifies the can juice and the dad. Then, an ID “kan-small-0001” is assigned to the can juice 21 and stored in the article data 611 in association with a pointer to the article attribute data 613 (see FIG. 5). At the same time, the article / mobile body search / management means 105 stores the article attribute data 613 of the can juice 21. Further, the article / mobile body search / management means 105 creates a “location history list 1” for storing the movement history of the can juice in the article history data 612.

  On the other hand, in order to update the father's moving body history data 622 detected by the sensing means 120, the article / moving body search / management means 105 refers to the position history of the ID “dad” in the moving body data 621. “Location history list 3” is called (see FIG. 6).

At time t1, the father sits at a position P4 (x4, y4) near the table and puts the can juice 21 he had on P1 (x1, y1, z1) on the table. When the in-environment sensing means 122 detects this, the article / moving body search / management means 105 sets the contents of the position history list 1 of the article history data 612 (see FIG. 5). In particular,
Time: t1
Details of handling: New Goods handling entity: Dad Position after handling: P1 (x1, y1, z1, α1, β1, γ1)
Set. Here, the handling content is “new” means that an article that has not existed in the environment until now is newly brought in from the outside.

Further, the article / mobile object search / management means 105 sets the contents of the position history list 3 of the mobile object history data 622 (see FIG. 6). In particular,
Time: t1
Position: (x4, y4, r1)
Condition: Sit down.

At time t2, the son (not shown), which is another moving body registered in the moving body data 621 of the moving body DB 62, uses the above can juice 21 placed in P1 (x1, y1, z1) on the floor. Move to P2 (x2, y2, z2) above. When the in-environment sensing means 122 detects this, the article / mobile body search / management means 105 receives the detection result and sets new contents of the position history list 1 of the article history data 612 (see FIG. 5). . In particular,
Time: t2
Handling contents: Moving Goods handling entity: Son Position after handling: P2 (x2, y2, z2, α2, β2, γ2)
Set. Here, the reason why the handling content is “move” is that an article already registered in the article history data 612 has been moved in the environment. Since the position of the son changes at time t2, the article / moving body search / management means 105 receives the detection result of the in-environment sensing means 122 and changes the contents of the son's moving body history data 622 (position history list 4). Although set, the illustration is omitted here.

At time t3, dad goes out of the room. When the in-environment sensing means 122 detects this, the article / mobile body search / management means 105 sets new contents of the position history list 3 of the mobile body history data 622 (see FIG. 6). In particular,
Time: t3
Position:-
Status: Set to go out. Here, the position is “−” because the in-environment sensing means 122 can no longer detect the father due to having gone out of the environment. In addition, as position information, you may set the position information of the door which passed when going out.

Further, when going out of the room, the father operates the operation terminal 103 to input a work instruction for moving the can juice 21 to P1 (x1, y1, z1). The robot 102 that has received the work instruction via the server 101 moves to the can juice position P2 (x2, y2) and holds the can juice at time t4. If the in-environment sensing means 122 detects this, the article / moving body search / management means 105 sets the contents of the moving body history data 622 (position history list 5) of the robot 102 (see FIG. 6). In particular,
Time: t4
Position: (x2, y2, r2)
State: [kan-small-0001]: Set the grip. The operation of the robot 102 may be detected by the in-environment sensing means 122, but the server 101 may detect the operation of the robot 102 by receiving the operation information from the robot 102.

At time t5, the robot 102 moves to P4 (x4, y4) while holding the can juice 21, and releases the can juice 21 held by P1 (x1, y1, z1) on the table. If the in-environment sensing means 122 detects this, the article / moving body search / management means 105 sets the new contents of the position history list 5 of the moving body history data 622 (see FIG. 6). In particular,
Time: t5
Position: (x4, y4, r2)
Status: [kan-small-0001]: Set release.

Further, the article / mobile body search / management means 105 sets the new contents of the position history list 1 of the article history data 612 (see FIG. 5). In particular,
Time: t5
Handling contents: Moving Goods handling entity: Robot Position after handling: P1 (x1, y1, z1, α1, β1, γ1)
Thereafter, the mother, which is still another moving body registered in the moving body data 621 of the moving body DB 62, enters the room from the door with the new can juice 22. When the entrance / exit sensing means 121 detects that the mother and the new can juice 22 have moved into the room, the article / mobile body search / management means 105 sends the new can juice 22 to the new can juice 22 [kan-small-0002]. ID is assigned and stored in the article data 611 in association with a pointer to the article attribute data 613. The article attribute data 613 of the new can juice 22 is the same as that of the can juice 21 described above. Further, a “location history list 2” for storing the movement history of the new can juice 22 in the article history data 612 is created (see FIG. 5).

At time t6, the mother places new can juice 22 on P3 (x3, y3, z3) on the floor. When the in-environment sensing means 122 detects this, the article / moving body search / management means 105 sets the contents of the position history list 2 of the article history data 612 (see FIG. 5). In particular,
Time: t6
Details of handling: New Goods handling entity: Mother Position after handling: P3 (x3, y3, z3, α3, β3, γ3)
Set. In FIG. 6, mom's moving body data and moving body history are omitted.

  As described above, the article / moving body search / management unit 105 stores / updates information on the article / moving body in the article / moving body DB 160. Here, an example is shown in which the database is updated every time the position of each article or moving body registered in the article / moving body DB 160 is changed. However, the updating timing of the database is not limited to this. What is necessary is just to set suitably.

(Environmental map management means / environmental map)
The environment map management means 107 creates the environment map 108 based on information from the sensing means 120 and manages the created environment map 108. This environment map 108 is used when the robot 102 moves in the environment. As will be described later, the robot 102 obtains the environment map 108 from the server 101 and makes a movement route plan.

  Further, when the environment map management means 107 receives an inquiry from the control means 110 to the environment map 108, the environment map management means 107 also sends necessary information to the control means 110 according to the content of the inquiry.

  For example, when the actual situation of the environment is shown in FIG. 8A, the environment map 108 may be obtained by simplifying the actual situation with a three-dimensional model (FIG. 8B). Further, as shown in FIG. 8C, the environment map 108 may be further simplified by a plane model. Further, an environment map 108 may be obtained by directly converting the actual situation of the environment shown in FIG. In other words, the environment map 108 may be created according to the purpose of use of the map and the time (labor) required to create the map. For example, when it is necessary to create an environment map composed of a three-dimensional model in a very short time, a three-dimensional object existing in the environment may be modeled with a minimum cuboid that covers the three-dimensional object. An environment map 108 shown in FIG. 8B is an example, and the table and the bookshelf are each modeled as a rectangular parallelepiped, and the trash can is modeled as a substantially cylinder. The same applies to the environment map composed of a planar model. In the work map 108 shown in FIG. 8C, the table and the bookshelf are respectively modeled by rectangular areas (areas with diagonal lines) projected onto the plane, and the trash can is represented as a circular area. Modeled with (hatched area). Further, in the map 108 composed of the planar model, the two rectangular areas and the circular area are set as areas where the robot 102 cannot move.

(Image management database / image management means)
The image management database 106 is a database in which the camera constituting the sensing means 120 accumulates and manages image data captured in the environment. This image management DB 106 is configured as shown in FIG. 9, and is composed of sub-databases that store image data 71 and recognition result data 72, respectively. The data contents stored in each sub-database are as follows.

  1) Image data 71: An image captured by the camera in the environment is stored in association with a time code for managing time. In this image data 71, it is not necessary to store images at all times taken by the camera. For example, an image is stored every certain time, an image is stored only when an event occurs, an image is stored at a specific time. It is only necessary to store only images in a necessary time zone by appropriately setting storage conditions such as not storing. The image storage unit 104 sets image storage conditions. The image stored in the image data 71 is deleted according to a predetermined condition. The deletion condition may be set such that the image is deleted when a certain time has elapsed after saving, an image related to a specific event is deleted, or the like. The image management means 104 also sets image deletion conditions. The image data 71 stores an image for each camera included in the system, and the image management unit 104 sets a camera that stores an image and a camera that does not store an image. The image data 71 stores information on the range captured by each camera (in the example shown, the camera 1 captures the bedroom and the camera 2 captures the living room), and the image management means 104 determines the image based on the information. In the data 71, a camera that stores an image and a camera that does not store are set.

  2) Recognition result data 72: A result of recognizing information obtained from an image stored in the image data 71 is stored. The recognition result data 72 stores a recognition result for each time code and for each camera. The recognition result is stored for each recognition level (information level). The recognition level means a result of recognizing information about a situation in the environment and information on a moving object and / or an article included in an image captured in the environment corresponding to the level of information included in the image. Information levels include, for example, 1) information on whether or not a moving object or article exists in the environment, and 2) information on the position of the moving object or article included in the image data in the image and the position in real space, 3) Information on attribute of moving body or article included in image data, 4) Information on posture of moving body or article, 5) Information on rectangular area surrounding area occupied by article and moving body in image, 6) Article in image And 7) information on the contour of the moving object, 7) information on the area of the face of the moving object in the image, 8) information on the movement path when the moving object moves, 9) information on the line-of-sight direction of the moving object, 10) movement For example, information on an article held by the body, 11) information on a region having the fastest moving speed in the image, 12) information on a region having the largest size in the image, and the like. In the illustrated example, “recognition level 1” is a result of recognizing a position in a real space (of a moving object or article), and “recognition level 2” is a result of recognizing an attribute of a detected object (article and moving object). It has become. Display control is performed using this recognition level as described later.

  As will be described later, the image management means 104 performs the above recognition process and stores the result in the DB. In addition, the image management means 104 sets which camera stores the recognition result recognized for the image data captured by the camera, and sets which information level is recognized at the time of the recognition. Or do.

(Control means)
The control means 110 in the environment management server 101 is a part that controls the entire server, and the main control contents are as follows.

  1) When the transmission / reception means 109 receives an inquiry about various data in the server, the inquiry content is judged, and data is sent to the article / mobile body search / management means 105 and the environment map management means 107 according to the judgment result. A reference request is issued.

  2) In response to the request, the result sent from the article / mobile body search / management means 105 and the environment map management means 107 is sent to the inquiry source via the transmission / reception means 109.

  3) The robot 102 interprets the work content message of the robot 102 transmitted from the operation terminal 103 via the transmission / reception means 109, generates a robot control command sequence for causing the robot 102 to execute an operation, and transmits it to the robot 102. The robot control command sequence will be described later.

  4) If necessary, the robot 102 and the user (operation terminal 103) can send the status of part or all of the articles managed by the article / mobile DB 160 and the status of the environment map 108 through the transmission / reception means 109 at regular intervals. ).

−Configuration of working robot−
The work robot 102, which is the second subsystem, carries out the work of handling articles according to work instructions specified by the user.

  As shown in FIG. 1, as a basic configuration of the robot 102, an obstacle sensor 111 that detects an obstacle in the vicinity of the robot 102, a gripping means 112 that grips an article, and an environment map 108 are used to make a movement plan. Plan creating means 113, moving means 114 for moving the robot 102 itself, transmission / reception means 109 for transmitting / receiving various data to / from the environment management server 101 and the operation terminal 103, these sensors 111, and each means 109, 112 ˜ And control means 115 for controlling 114.

  FIG. 10 is a schematic diagram showing an example of the structure of the robot 102 in the present system. The robot 102 includes a substantially box-shaped main body 10 that accommodates the movement plan creation means 113, the control means 115, and the like. Hereinafter, the right side of FIG. 10 is referred to as the front side, the left side is referred to as the rear side, the back side of the paper surface is referred to as the left side, and the front side of the paper surface is referred to as the right side.

  The gripping means 112 is composed of an articulated arm 12a and a hand 12b disposed at the tip of the arm 12a, and is attached to the upper surface of the main body 10. The arm 12a and the hand 12b may be ones using motor-controlled actuators or other actuators such as artificial muscle actuators.

  The moving means 114 is constituted by wheels 14, and two wheels 14 are attached to the left and right sides of the main body 10 (the left wheel is not shown in the figure). Here, the moving means 114 is composed of the wheels 14, but the structure of the moving means 114 may be selected in accordance with the environment in which the robot is used. For example, when the undulation of the environment floor is severe, it is preferable to configure the moving means 114 as a crawler type or a multi-legged walking type.

  In the present system, the obstacle sensor 111 includes an ultrasonic sensor 11a, a stereo camera 11b as a visual sensor, and a collision sensor 11c.

  The ultrasonic sensor 11a calculates the distance from the sensor 11a to the obstacle by measuring the time until the reflected wave is received after emitting the ultrasonic wave. It is for detecting before doing. Three ultrasonic sensors are attached to each side surface (front surface, rear surface, left and right side surfaces) of the main body.

  The stereo camera 11b inputs the situation around the robot 102 as an image and performs processing such as image recognition to obtain more accurate information on the object to be grasped and the presence of an object to be grasped. It is. The stereo camera 11b is attached to the front part of the main body part 10.

  The collision sensor 11c is a sensor that detects that a predetermined impact force is applied to the robot 102. For example, the collision sensor 11c detects that an obstacle has collided with the robot 102 or that the robot 102 has collided with the obstacle while moving. The collision sensor 11c is attached to each of the front surface and the rear surface of the main body.

  The movement plan creation means 113, when the movement movement of an article or other movement is designated to the robot 102, shows the movement path from the current position of the robot 102 to the designated position (target position). It is created using the map 108. At this time, there should be no obstacle on the movement path from the current position to the target position, but the environment map 108 has an area in which the robot cannot move (for example, a hatched line in FIG. 8C). Area) is set. For this reason, if a movement route is created in an area other than the unmovable area, a movement path that avoids an obstacle is created. For example, when the robot is moved from the A1 point to the A2 point using the environment map 108 using the planar model of FIG. 8C, a route for avoiding the immovable region in consideration of the size of the robot 102 (FIG. 8 (c) is created). In creating such a movement route, the Dijkstra method, which is the most general method, may be used. If the environment is complicated, a route search algorithm improved from the Dijkstra method may be used.

  The control means 115 of the work robot 102 mainly interprets a robot control command sequence sent from the environment management server 101 via the transmission / reception means 109, and executes the robot control commands in order.

  Here, the robot control command is a command for controlling the gripping of the article and the movement of the robot 102 itself. In this embodiment, the robot control command moves from the current position of the robot 102 to the position specified by coordinates. Three types are set: “move” which is a command, “gripping” which is a command for gripping an article specified by the article ID by the hand 12b, and “release” which is a command for releasing the hand 12b. The robot control commands are not limited to these three types, and may be increased as necessary.

-Configuration of operation terminal-
An operation terminal 103, which is the third subsystem, is a user interface in this system, and is a terminal operated by the user in order to mainly instruct the handling operation of the article with respect to the robot 102. The operation terminal 103 displays an image captured by the camera in response to a user request. Thereby, the user can confirm the state in each environment.

  As shown in FIG. 1, the operation terminal 103 is composed of, for example, a CRT or a liquid crystal display for displaying an operation screen for designating work contents of the robot 102 and a monitoring screen for displaying an image captured by the camera. A display unit 117, an input unit 116 composed of, for example, a pointing device or a keyboard for performing input on the operation screen, a display control means 118 for creating a monitoring screen displayed on the display unit 117, and display control thereof Display control database 123 for accumulating data for transmission, transmission / reception means 109 for transmitting the work contents of the robot 102 input by the input unit 116 to the environment management server 101, and control for controlling these means 109, 116-118. Means 119.

  For example, a general-purpose PC can be used as the operation terminal 103. In this case, the PC can be used as the operation terminal 103 when the PC reads the control program.

  The user designates the work content of the robot 102 using the operation terminal 103, and confirms the situation in the environment as will be described later. Further, the user inputs information on articles and moving objects stored in the article / moving object management DB 160 using the operation terminal 103 for managing articles / moving objects, The search for the current position is also requested. The search result of the current position of the article is sent from the server 101 and displayed on the display unit 117 of the operation terminal 103.

  As described above, the display control DB 123 stores data for creating a monitoring screen. The display control DB 123 is configured as shown in FIG. 11, and is composed of sub-databases that store three types of data, status data 81, access right data 82, and display control data 83, respectively. The data contents stored in each sub-database are as follows.

  1) Status data 81: The status content in the environment related to the processing of the image data is stored, and each status content is associated with the status ID. Further, “explanation” is provided in association with the situation ID, but this “explanation” is to explain the situation content and can be omitted.

  Here, the situation content is expressed by a moving object, movement of the moving object, article, location (environmental attribute), time, climate, and a combination thereof. Examples of moving objects include "Father", "Children", "Men", "Visitors", and "Pets". The actions of moving objects are "sleeping", "mealing", "carrying things", "seeing clothes", "absence" “, Futon” and “can juice” are illustrated as articles, “bedroom” “bath” “table” “in front of TV” are illustrated as places, and “9 o'clock” is illustrated as time -10:00 "," Wednesday "," Holiday "are exemplified, and" rain "" 20 degrees or more "is exemplified as the climate. By combining these, as shown in FIG. 11, “person; adult, place; bedroom, article; futon” is set as the situation content representing the sleeping situation. Each of these situations is acquired from the detection result of the sensing means 120, the information stored in the article / moving body DB 160, and the recognition result of the image data.

  2) Access right data 82: Users accessing the system (users using the system) are classified and stored for each access right (access users). An access ID is added to each access user. The access user may be an individual (in the illustrated example, father, mother), or may be a group including individuals having the same attributes (in the illustrated example, children. In addition, adults, men, and women).

  3) Display control data 83: Corresponding to the display ID, the processing contents and processing conditions of the image are stored. As will be described later, when the processing condition is satisfied, the display control unit 118 processes the image data according to the processing content corresponding to the processing condition and creates a monitoring screen.

  In the processing condition, the situation ID and the access ID are described. In addition, the processing content describes the specific content of processing applied to the image requested by the user. As processing contents, for example, 1) Mosaic processing is performed on the face area using the result of recognizing the face area of the moving object in the image. 2) The result of recognizing the attribute of the moving object and the movement in the image Using the result of recognizing the position of the body, a process for masking the user's family in the image (a process in which a person other than the family is included in the image), 3) a rectangular area surrounding the area occupied by the moving body in the image is recognized A process for replacing a person included in an image with an icon using the result, 4) A process for masking an object held by a person in the image using a result of recognizing an article held by the moving object, 5 ) Processing for creating a map showing the position of the moving object using the result of recognizing the position of the moving object in the real space, 6) In the environment using the result of recognizing the attribute of the moving object existing in the environment Moving body In a one of the information to create a screen including the (text information) only (for masking the entire screen) process, etc. are exemplified. In the example shown in the drawing, processing for replacing a person with an icon is described as the processing content of the display ID: 0002.

-Information provision processing-
Next, an information providing process in the information providing system having the above configuration will be described. This information providing process includes an image recognition process for accumulating a camera image and a recognition result of the camera image in a database, and an image presentation process for presenting an image processed according to the situation in the environment and the access user to the user. Including.

(Image recognition processing)
FIG. 12 shows a flowchart of image recognition processing. First, in step S1301, images taken by all cameras (sensing means 120) are input. In step S1302, the image management unit 104 stores the camera image with the time code added to the image data 71 of the image management DB 106 in accordance with a preset storage condition. As described above, the storage condition is set such that an image is stored at regular intervals, an image is stored only when an event occurs, an image is not stored at a specific time, and the like. The image management means 104 uses information stored in the article / mobile object DB 160 as necessary.

  In step S1303, the image management unit 104 performs recognition processing for each image stored in step S1302. In this recognition, the information stored in the article / moving body DB 106 is used as appropriate, and sensing by the sensing means 104 is executed as necessary.

  In step S1304, the image management unit 104 stores the recognized result for each recognition level in the recognition result data 72 of the image management DB 106 according to the camera condition and the recognition level condition for storing the recognition result set in advance. To do. Note that the processing in step S1303 and step S1304 is executed for all camera images stored in the image data 71 and for all recognition levels registered in the recognition result data 72. Thus, the image management DB 106 stores the image data captured by the camera and the recognition result of the image data in association with each other.

(Image presentation processing)
FIG. 13 shows a flowchart of the image presentation processing. First, in step S1401, the user requests the system to view the camera image via the operation terminal 103. At this time, the user designates at least the environment (room) to be browsed. Moreover, you may further specify conditions, such as a mobile body to browse, operation | movement of a mobile body, time, and a climate.

  In subsequent step S1402, the display control means 118 identifies the user who has accessed the system. Prior to the operation at the operation terminal 103, user authentication may be performed using an ID and a password, or the user operating the operation terminal 103 may be specified based on the detection result of the sensing means 120. Good (information on the user operating the operation terminal 103 specified based on the detection result of the sensing means 120 is stored in the article / moving body DB 160).

  In step S1403, a camera image obtained by capturing the environment related to the browsing request is extracted from the image management DB 106, and the situation in the environment corresponding to the extracted camera image is confirmed. This is confirmed based on information stored in the image management DB 106, information stored in the article / mobile object DB 160, and information newly detected by the sensing means 120 as necessary.

  In step S1404, the display control means 118 specifies the access ID based on the user information acquired in step S1402, and specifies the situation content (situation ID) based on the situation confirmed in step S1403. Specifically, the access right data 82 in the display control DB 123 is searched, and the access ID corresponding to the user (access user) identified in step S1402 is extracted. Further, the situation data 81 in the display control DB 123 is searched to extract the situation ID corresponding to the situation confirmed in step S1403.

  Thus, when the access ID and the situation ID are extracted, the display ID corresponding to the combination of the access ID and the situation ID is extracted by referring to the display control data 83 of the display control DB 123.

  In step S1405, the display control means processes the image using the recognition result in accordance with the processing content corresponding to the display ID extracted in step S1404, and displays a monitoring screen including the processed image on the display unit 117. To display.

  Next, the image recognition process and the image presentation process will be described with specific examples. FIG. 14 shows the configuration of the image management DB 106 and the display control DB 123 according to a specific example.

  First, as described above, when an image captured by the camera constituting the sensing unit 120 is input, the image management unit 104 stores the camera image in the image management DB 106 in accordance with the set storage conditions. Here, it is assumed that a storage condition of “save images of all cameras at 1 second intervals” is set, and accordingly, images captured by the camera 1 and the camera 2 are associated with time codes and imaged. The image data 71 is stored in the management DB 106 (see FIG. 14). Of these, the camera 2 is a camera that captures images in the living room, and the image shown in FIG. Note that the time code 0000003 corresponds to the time zone after the meal.

  Simultaneously with the accumulation of the camera image, based on the detection result of the article and the moving body by the sensing means 120, the information of the article and the moving body in the environment imaged by the camera is accumulated in the article / moving body DB 160.

  The image management means 104 performs recognition processing on the images stored in the image data 71. Here, since recognition level 1 “rectangular area of the moving object in the image” and recognition level 2 “attribute of the moving object” are set in the recognition result data 72, the image management means 104 performs the recognition level 1. , 2 is performed. The recognition level 1 “rectangular region of the moving object in the image” is obtained by calculating a rectangle circumscribing the difference region in the background difference image. This circumscribed rectangle can be represented by the coordinates “(X1, Y1) − (X2, Y2)” of the two diagonals of the rectangle (see FIG. 14). Also, the “level attribute of the moving object” at the recognition level 2 is obtained from information stored in the article / moving object DB 160. Thus, the recognition level 1 “(X1, Y1)-(X2, Y2)” and the recognition level 2 “Father” are stored in the recognition result data 72 as the recognition result of the time code 0000003 of the camera 2, respectively.

  In this state, it is assumed that the child requests the system to view a camera image captured in the living room via the operation terminal 103 in order to “know the state of the living room”.

  The display control means 118 refers to the article / moving body DB 160 and identifies the user who has accessed. Thereby, it is specified that the access user is “child”.

  Next, the display control means 118 extracts the image of the camera 2 stored in the image data 71 since the user designates the camera image taken in the living room. The situation in the living room corresponding to this image is confirmed based on the information in the image management DB 106 and the article / moving body DB 160. Since the recognition level 2 in the recognition result data 72 of the image management DB 106 is recognized as “father”, it is confirmed that the father is present in the living room. Since the time code 0000003 corresponds to the time zone after the meal, the situation corresponding to the image of the camera 2 with the time code 0000003 stored in the image data 71 is “father”, “living room”, and “after meal”. is there.

  Then, the display control means 118 specifies an access ID based on the identified user. Since the access user is “child”, the access ID is “000003”. Further, the situation ID is specified based on the situation corresponding to the image. Since the situation corresponding to the image of the camera 2 with the time code 0000003 is “father”, “living room”, and “after meal”, the situation ID is “0002”.

  When the access ID and the situation ID are specified in this way, the display control means 118 extracts the display ID corresponding to the combination of the access ID and the situation ID from the display control data 83 of the display control DB 123. Here, since the situation ID is 0002 and the access ID is 000003, the display ID is 0002.

  The image of the camera 2 with the time code 0000003 is processed according to the processing content corresponding to the display ID thus extracted. Here, since “replace a person with an icon” is set as the processing content, data of recognition level 1 “rectangular area in the image” in the recognition result data 72 is used to display an icon in the rectangular area. Process to paste. In this way, the image shown in FIG. 15 is created and displayed on the display unit 117 of the operation terminal 103. In the processing of the image, information of the recognition level 2 may be used to display the attribute of the person (here, “Father”) on the icon.

  As described above, the information providing system according to the present invention is a system that provides information related to a situation in a predetermined space, and includes an imaging unit (sensing unit 120) that images the inside of the space, and a space that is captured by the imaging unit. Identification means (article / moving body search management means 105 and display control means 118) for identifying the situation in the image, and recognition means for recognizing information about the moving body and / or the article included in the image data captured by the imaging means (The image management means 104), the processing content management database (display control database 123) for storing the situation in the space and the processing content of the image data in association with each other, and the processing content management database, with reference to the identification Processing means (display) that processes the image data captured by the imaging means using the recognition result of the recognition means according to the processing content corresponding to the situation specified by the means Comprising a control means 118), display means for displaying the processed image data (the operation terminal 103), the.

  With this configuration, it is possible to change the content and level of information to be provided by processing captured image data according to the situation in the space. Accordingly, unnecessary information can be deleted from the image data while including necessary information in the image data, and flexible information corresponding to the situation can be realized when providing information about the situation in the space.

  The information providing system further includes an operation means (operation terminal 103) for outputting a display request signal for image data in response to a user operation. The processing content management database includes user information, a situation in the space, and image data. The processing means receives the display request signal, acquires the user information related to the output of the display request signal, and determines the user information and the status specified by the specifying means. The image data is processed according to the corresponding processing content. Thereby, in the combination of the user who browses the image data and the situation in the space, appropriate information can be provided to the user.

  The processing content management database stores a combination of an attribute of the space and an attribute of a moving object and / or an article existing in the space as a situation in the space.

  Since the situation in the space is specified by a combination of the attribute of the space and the attribute of the moving object and / or the article, more appropriate information can be provided. For example, in the case of a general household, when the attribute of the space is a bathroom, the information of the moving object existing in the bathroom is deleted by processing the image data, and when the attribute of the space is a living room, Including information on existing mobile objects is realized.

  The information providing system further includes sensing means 120 for detecting a moving object and / or article existing in the space, and the specifying means specifies and recognizes the situation in the space based on the detection result of the sensing means. The means recognizes information about the moving object and / or the article included in the image data based on the image data picked up by the image pickup means and / or the detection result of the sensing means.

  With this configuration, the specifying unit can accurately specify the situation in the space, and the recognizing unit can accurately recognize information about the moving object and / or the article included in the image data.

  The recognizing means recognizes information about the moving object and / or the article for each of a plurality of information levels, and the processing content management database includes a situation in the space and a recognition result of the information level selected according to the situation. The processing contents set to be included in the processed image data are stored in association with each other.

  In this way, the information (information level) included in the image data can be changed more finely, and it is possible to appropriately delete unnecessary information from the image data while including the necessary information in the image data. obtain.

  The information providing system further includes an image management database 106 that stores the image data captured by the imaging unit and the recognition result recognized by the recognition unit in association with each other, and the processing unit stores the image stored in the image management database. Data is processed using recognition results stored in the image management database.

  By doing so, the image data picked up by the image pickup means is stored in the image management database, so that it is possible to view the image data before processing as necessary.

(Other embodiments)
In the above embodiment, the information providing system is composed of the three subsystems of the environment management server 101, the robot 102, and the operation terminal 103. These subsystems 101 to 103 communicate with each other via a wireless or wired network. It was set as the structure which exchanges. However, the information providing system is not limited to this configuration, and for example, a configuration in which the operation terminal 103 is integrated with the environment management server 101 may be used. In other words, the environment management server 101 may include the input unit 116, the display unit 117, the display control unit 118, and the display control database 123. The robot 102 may be omitted.

  In the above embodiment, the image data and the recognition result are accumulated in the image management database 106, and the display control means 118 processes the image data accumulated in the image management database 106 using the recognition result. However, the image management database 106 may be omitted, and the image data captured by the sensing means 120 may be sequentially processed to display the processed image.

  Furthermore, in the above embodiment, the present system is applied to general households, but it can be applied to various other spaces such as daycare, nursing care facilities, hospitals, offices, hotels and stores. These can be used for space monitoring. Further, in the present embodiment, the purpose is mainly to protect the privacy of the moving body, but the present system can be aimed at ensuring information security, for example. Specifically, when a confidential document is included in image data captured in the space, information leakage can be prevented by performing mask processing on the confidential document area in the image data. This can be applied to an office, for example.

  As described above, the present invention can easily change the content and level of information to be provided by processing image data according to the situation in the space. It is useful for systems that provide information about the situation in various spaces such as hospitals, offices, hotels and stores.

It is a block diagram which shows an example of the whole structure of the information provision system which concerns on this embodiment. It is a figure which shows the example which has arrange | positioned the gate-type reader / writer in the door and window in an environment. It is explanatory drawing which shows the principle of a background difference method. It is a block diagram which shows the structure of an article / mobile body search / management means. It is a figure which shows the structure of an article database, and the example of the description content. It is a figure which shows the structure of a mobile body database, and the example of the description content. It is explanatory drawing explaining an example in which articles | goods move within an environment. It is a figure which illustrates the actual condition of an environment, and the environment map corresponding to it. It is a figure which shows the structure of an image management database, and its description content example. It is the schematic which shows an example of a structure of a working robot. It is a figure which shows the structure of a display control database, and its description content example. It is a flowchart of an image recognition process. It is a flowchart of an image provision process. It is a figure which shows the description content of the image management database and display control database which concern on a specific example. It is an example of the processed image data.

Explanation of symbols

101 Environmental management server (information provision server)
103 Operation terminal (display means, operation means)
104 Image management means (recognition means)
105 Article / mobile object search management means (specifying means)
106 Image management database
110 Control means
116 Input section
117 Display
118 Display control means (specifying means, processing means)
120 Sensing means
123 Display control database (Processing content management database)
160 Article / Moving Object Database

Claims (8)

An information providing system for providing information on a situation in a predetermined space,
Imaging means for imaging the space;
Specifying means for specifying the situation in the space imaged by the imaging means;
Recognizing means for recognizing information about a moving object and / or an article included in the image data captured by the imaging means;
A processing content management database that stores the situation in the space and the processing content of the image data in association with each other;
Processing means for processing image data captured by the imaging means using the recognition result of the recognition means according to the processing content corresponding to the situation specified by the specifying means with reference to the processing content management database;
An information providing system comprising: display means for displaying the processed image data. In the information provision system of Claim 1,
Further comprising an operation means for outputting a display request signal for image data in response to a user operation,
The processing content management database stores the user information and the situation in the space in association with the processing content of the image data,
The processing means receives the display request signal, obtains user information related to the output of the display request signal, and processes the image data according to the processing content corresponding to the user information and the situation specified by the specifying means. Offer system. In the information provision system of Claim 1,
The processing content management database is an information providing system that stores a combination of an attribute of the space and an attribute of a moving object and / or an article existing in the space as a situation in the space. In the information provision system of Claim 1,
Sensing means for detecting a moving object and / or an article existing in the space;
The specifying means specifies the situation in the space based on the detection result of the sensing means,
The recognition means is an information providing system for recognizing information about a moving object and / or an article included in the image data based on the image data picked up by the image pickup means and / or the detection result of the sensing means. In the information provision system of Claim 1,
The recognition means recognizes information about the moving object and / or the article for each of a plurality of information levels,
The processing content management database associates and stores the situation in the space and the processing content set so that the recognition result of the information level selected according to the situation is included in the processed image data . In the information provision system of Claim 1,
An image management database for storing the image data captured by the imaging unit and the recognition result recognized by the recognition unit in association with each other;
An information providing system for processing the image data stored in the image management database using the recognition result stored in the image management database. An information providing server that provides information on a situation in a predetermined space,
A specifying means for specifying the situation in the space imaged by the imaging means;
Recognizing means for recognizing information about the moving object and / or article included in the image data captured by the imaging means;
By referring to the processing content management database that stores the situation in the space and the processing content of the image data in association with each other, according to the processing content corresponding to the situation specified by the specifying unit, the recognition result of the recognition unit is used. Processing means for processing image data captured by the imaging means;
An information providing server comprising: output means for outputting the processed image data. An information providing method for providing information on a situation in a predetermined space,
An imaging step of imaging the space;
A specific step of identifying the situation in the space imaged in the imaging step;
A recognition step for recognizing information about a moving object and / or article included in the image data captured in the imaging step;
A processing step for processing the image data captured in the imaging step using the recognition result of the recognition step according to the processing content set corresponding to the situation specified in the specific step,
A display step for displaying the image data processed in the processing step.

Images