JP5357966B2 - Information system, server device, terminal device, information processing method, and program - Google Patents

Abstract

To solve the problem that identifying imaged objects requires a large amount of time, a terminal device is disclosed that is provided with: a camera unit that stores at least one object information item and acquires imagery, said object information item comprising an object identifier and region information for an object; a terminal information acquisition unit that acquires terminal information comprising position information and direction information; an imaged object decision unit that applies the terminal information and a view angle to the region information in each object information item, examines each imaged object in ascending order of distance from the device, marks information for regions that become invisible due to object imaging as invisible regions, and stops examining when there are no more visible regions; an imaged object information acquisition unit that acquires at least one decided imaged object information item; and an output unit that outputs the imagery and at least one imaged object information item. This makes it possible to identify objects imaged by a camera and output information pertaining to said objects.

Description

  The present invention relates to an information system that recognizes an object photographed by a camera and displays object information on a screen.

  Conventionally, information distribution type landscape labeling in which geographical information on a computer is associated with each part in a landscape image of a real scene and presented to the user, and various information (advertisement information, guidance information, etc.) is sent to the user. There was a device (Patent Document 1).

  In addition, the background data processing, data transfer, and terminal display time on the host computer can be greatly reduced, and 3D map information using the latest background map can be configured on the terminal device to perform light operations. There was a map-related information distribution system that can be used (Patent Document 2).

Furthermore, there has been a subject information processing apparatus and a subject information processing method that automatically embed subject information in a video photographed by a camera device (Patent Document 3).
JP-A-11-66347 JP 2008-158583 A JP 2002-152713 A

  However, the conventional system does not assume that a large number of objects are registered. That is, when a very large number of objects are registered, it takes a lot of time to specify the object currently being shot, and a great amount of CPU power is required. Therefore, for example, it cannot be used for a navigation system.

  The information system according to the first aspect of the present invention is an information system including a server device and a terminal device, and the server device includes an object identifier for identifying an object and region information indicating a region where the object exists. An object information storage unit that can store one or more pieces of information, and a server reception unit that receives terminal information including position information indicating the position of the terminal device and direction information indicating a direction in which the terminal device is facing from the terminal device; Applying the position information included in the terminal information, the direction information included in the terminal information, and the viewing angle stored in advance or included in the terminal information to the area information of each object information in the object information storage unit. A shooting object determination unit that determines one or more objects being shot, and one or more determined by the shooting object determination unit Of one or more object information corresponding to each object, a shooting object information acquisition unit that acquires one or more shooting object information including area information from the object information storage unit, and a server that transmits the shooting object information to the terminal device A terminal unit that acquires terminal information including position information indicating a position of the terminal device and direction information indicating a direction in which the terminal device is facing, and a video. Corresponding to the transmission of the terminal unit, the output unit including the camera unit, the camera image output means for outputting the video acquired by the camera unit, the transmission unit for transmitting the terminal information acquired by the terminal information acquisition unit to the server device A receiving unit that receives one or more shooting object information, and one or more area information included in the one or more shooting object information received by the receiving unit. An output information acquisition unit that acquires output information, and the output unit outputs region information included in the one or more output information acquired by the output information acquisition unit, overlaid on the video output by the output unit And the shooting object information determination unit inspects the objects in order from the shortest distance from the terminal device, and becomes invisible by shooting the object previously determined that the terminal device is shooting. This is an information system for registering area information as an invisible area and ending the inspection of whether or not an object is photographed by the terminal device when the visible area disappears.

  With this configuration, it is possible to perform high-speed processing by appropriately terminating the processing for detecting a shooting object. Therefore, it can also be used for navigation systems.

  In the information system of the second invention, on the other hand, the area information includes two-dimensional area information indicating a two-dimensional area on the map and height information indicating the height of the object. The shooting object determination unit acquires the shooting area on the three-dimensional space using the position information and direction information included in the terminal information and the viewing angle stored in advance or included in the terminal information, This is an information system that is applied to the area information included in each object information of the object information storage unit and determines one or more objects that are photographed by the terminal device.

  With such a configuration, an object photographed by a camera can be captured in three dimensions. Therefore, for example, it is possible to recognize a building that is higher than the previous building and is reflected in the camera image, and output information related to the building. In addition, high-speed processing is possible by appropriately terminating the processing for detecting a shooting object. Therefore, it can also be used for navigation systems.

  In the information system according to the third aspect of the present invention, in the first or second aspect, the region output means has a region corresponding to the region information included in the one or more pieces of output information acquired by the output information acquisition unit. Is an information system that outputs the image on the video output by the output unit so that it can be visually distinguished from other regions.

  With this configuration, it is possible to output the area of the object photographed by the camera and displayed on the display in accordance with the size of the object.

  In the information system according to the fourth aspect of the present invention, in any one of the first to third aspects, the server device determines the distance between each object determined by the shooting object determination unit and the terminal device and the object information of each object. The camera further includes a distance calculation unit that calculates each object determined by the shooting object determination unit from the position information included in the terminal information, and the shooting object information acquisition unit corresponds to one or more objects determined by the shooting object determination unit. An information system that acquires information including region information from one or more object information to be obtained from an object information storage unit, and configures one or more shooting object information including information including region information and a distance calculated by a distance calculation unit It is.

  With such a configuration, it is possible to acquire the distance to the object displayed on the display by photographing with the camera.

  In the information system according to the fifth aspect of the invention, in contrast to the fourth aspect, the output information acquisition unit outputs output information including one or more area information and one or more distances included in the shooting object information received by the receiving unit. The acquisition and output unit is an information system further comprising distance output means for outputting a distance included in the one or more pieces of output information acquired by the output information acquisition unit.

  With this configuration, it is possible to output the distance to the object that is captured by the camera and displayed on the display.

  In the information system of the sixth aspect of the invention, in contrast to the fourth or fifth aspect, the region output means corresponds to the distance as the distance included in the one or more pieces of output information acquired by the output information acquisition unit increases. An information system that outputs an object with a smaller area.

  With this configuration, it is possible to output the area of the object photographed by the camera and displayed on the display to the terminal device in accordance with the size of the object.

  The information system of the seventh invention further comprises one or more web server devices for any of the first to sixth, and the web server device stores object related information that is information about the object. An object-related information storage unit, an object-related information acquisition unit that acquires object-related information in response to an external instruction, and an object-related information transmission unit that transmits object-related information. The information system further includes an object related information receiving unit that receives related information, and the output unit further includes an object related information output unit that outputs the object related information received by the object related information receiving unit.

  With this configuration, it is possible to output information related to the object that is captured by the camera and displayed on the display.

  Further, in the information system of the eighth invention, in contrast to the seventh, the terminal device includes a first display unit and a second display unit which are two separated display units, and the camera image output unit includes: The video acquired by the camera unit is displayed on the first display unit, and the object-related information output unit is an information system that displays the object-related information received by the object-related information receiving unit on the second display unit.

  With this configuration, it is possible to output both video captured by the camera and information related to objects in the video to appropriate areas.

  In the information system according to the ninth aspect of the present invention, the area output means does not output the area information of another object, but only the area information of one object is output to the other area. This is an information system that outputs the image superimposed on the video output by the output unit so that it can be visually distinguished from the information.

  With this configuration, it is possible to appropriately output the object area in the video image captured by the camera.

  In the information system according to the tenth aspect of the invention, for any of the seventh to ninth aspects, the object related information output means does not output the object related information of other objects, but only the object related information of one object. Is an information system that outputs

  With this configuration, it is possible to appropriately output information related to the object in the video being shot by the camera.

  Further, in the information system of the eleventh aspect of the invention, in the ninth or first aspect, the shooting object determination unit is a shooting object of an object that is present at the closest position in the video shot by the camera unit, Or an information system for determining the shooting object of the object located closest to the center in the video shot by the camera unit or the shooting object of the object having the largest area in the video shot by the camera unit is there.

  With this configuration, it is possible to output information or a region related to one object in a video shot by the camera.

  Also, in the information system of the twelfth aspect of the invention, for any one of the first to the first, the terminal device outputs a first receiving means for receiving a user instruction, and an output when the receiving section receives the instruction. The information system further includes an image being output by the unit and an output fixing unit for fixing the area.

  With such a configuration, since the video imaged by the camera can be fixed and fixed, the operation on the object in the video becomes easy.

  Further, in the information system of the thirteenth invention, in contrast to the first, the shooting object information includes an object identifier, and the terminal device includes a second receiving unit that receives an input to the fixed area, An object identifier acquisition unit that acquires an object identifier for an area corresponding to the input received by the reception unit; and an object identifier transmission unit that transmits the object identifier acquired by the object identifier acquisition unit to one of the one or more web server devices. Further, the object related information receiving unit is an information system that receives object related information in response to transmission of an object identifier, and the transmission of the object identifier is performed by an instruction from the outside.

  With such a configuration, since the video imaged by the camera can be fixed and fixed, the operation on the object in the video becomes easy.

  In the information system according to the fourteenth aspect of the present invention, in any of the eighth to first thirteenth aspects, the terminal device has a camera layer, which is a layer for outputting video acquired by the camera unit, and an area corresponding to the object. A current layer storage unit that can store a current layer that is a current layer among a plurality of layers including an object identification layer that is an output layer and a web search layer that is a layer that outputs object-related information, and a current layer A third receiving means for receiving an input to be set; and a current layer changing section for changing the current layer of the current layer storage section according to the input received by the third receiving means, and when the current layer is a camera layer, The camera video output means outputs the video acquired by the camera unit, and the current layer is the object In the case of another layer, the camera video output means outputs the video acquired by the camera section, and the area output means outputs the video output by the output section so that the area can be visually distinguished from other areas. When the current layer is a web search layer, the camera video output means outputs the video acquired by the camera unit, and the area output means can visually distinguish the area from other areas. The object related information output means is an information system that outputs the object related information by superimposing it on the video output by the output unit.

  With such a configuration, it is possible to display appropriate information for an object in a video image taken by the camera.

  In the information system according to the fifteenth aspect of the present invention, the object related information is one or more information of a web page, a symbol image of the object, or sound information related to the object. Is an information system.

  With such a configuration, it is possible to display appropriate information for an object in a video image taken by the camera.

  The information system of the sixteenth invention is an information system comprising a server device and a terminal device, wherein the server device includes an object identifier for identifying an object, region information indicating a region where the object exists, A server that receives, from the terminal device, terminal information having position information indicating the position of the terminal device and direction information indicating the direction in which the terminal device is facing. Applying the receiving unit, the position information included in the terminal information, the direction information included in the terminal information, and the viewing angle stored in advance or included in the terminal information to the area information of each object information in the object information storage unit, A shooting object determining unit that determines one or more objects being shot by the terminal device and a shooting object determining unit are determined. Of the one or more object information corresponding to one or more objects, one or more shooting object information including an object identifier is acquired from the object information storage unit, and the shooting object information is provided to the terminal device. A server transmission unit for transmitting, and the terminal device acquires terminal information including position information indicating a position of the terminal device and direction information indicating a direction in which the terminal device is facing, and a video A camera unit for acquiring the image, an output unit including a camera video output unit for outputting the video acquired by the camera unit, a transmission unit for transmitting the terminal information acquired by the terminal information acquisition unit to the server device, and transmission of the terminal information Corresponding to the receiving unit, and a receiving unit that receives the shooting object information, and an output unit that outputs the shooting object information received by the receiving unit. That is an information system.

  With such a configuration, it is possible to display appropriate information for an object in a video image taken by the camera.

  Further, the information system according to the seventeenth aspect of the present invention further comprises at least one web server device as compared with the sixteenth, and the web server device can store object related information that is information related to an object. An information storage unit, an object-related information acquisition unit that acquires object-related information in response to an instruction from the outside, and an object-related information transmission unit that transmits object-related information, and the server transmission unit of the server device includes: The shooting object information is an information system that transmits the shooting object information to the terminal device or one or more web server devices, and is received by the receiving unit of the terminal device.

  With such a configuration, it is possible to display appropriate information for an object in a video image taken by the camera.

  The terminal device according to the eighteenth aspect of the present invention includes an object information storage unit capable of storing one or more object information having an object identifier for identifying an object and region information indicating a region in which the object exists, and a terminal device Terminal information acquisition unit for acquiring terminal information having position information indicating the position of the terminal device and direction information indicating the direction in which the terminal device is facing, position information included in the terminal information acquired by the terminal information acquisition unit, and terminal information A shooting object determination unit that applies the direction information and the viewing angle stored in advance to the area information of each object information in the object information storage unit, and determines one or more objects being shot by the terminal device; The region information is included in one or more object information corresponding to one or more objects determined by the shooting object determination unit. A shooting object information acquisition unit that acquires the above shooting object information from the object information storage unit, a camera unit that acquires a video, a camera video output unit that outputs a video acquired by the camera unit, and a video 1 It is a terminal device comprising an output unit comprising region output means for outputting region information included in the above photographing object information.

  With such a configuration, it is possible to display appropriate information for an object in a video image taken by the camera. Also, appropriate information can be displayed easily without communication with the server device.

  In addition, the terminal device according to the nineteenth aspect of the present invention, in contrast to the eighteenth aspect, has region information including two-dimensional region information indicating a two-dimensional region on a map and height information indicating the height of an object. Then, the shooting object determination unit acquires the shooting area in the three-dimensional space by using the position information and direction information included in the terminal information and the pre-stored viewing angle, and stores the shooting area in the object information. This is a terminal device that is applied to the area information included in each piece of object information and determines one or more objects being photographed by the terminal device.

  With such a configuration, an object photographed by a camera can be captured in three dimensions.

  Further, in the terminal device of the twentieth aspect of the invention, in contrast to the eighteenth or nineteenth aspect, the region output means has a size corresponding to region information included in one or more pieces of output information acquired by the output information acquisition unit. This is a terminal device that superimposes the area on the video output by the output unit so that it can be visually distinguished from other areas.

  With this configuration, it is possible to output the area of the object photographed by the camera and displayed on the display in accordance with the size of the object.

  The terminal device according to the twenty-first aspect of the present invention provides the distance between each object determined by the shooting object determination unit and the terminal device with respect to any one of the eighteenth to the second, the object information of each object, and the terminal. The camera further includes a distance calculation unit that calculates each object determined by the shooting object determination unit from position information included in the information, and the shooting object information acquisition unit corresponds to one or more objects determined by the shooting object determination unit. A terminal device that obtains information including area information from one or more pieces of object information from the object information storage unit and constitutes one or more shooting object information including information including area information and the distance calculated by the distance calculation unit. is there.

  With such a configuration, it is possible to acquire the distance to the object displayed on the display by photographing with the camera.

  The terminal device of the twenty-second invention is a terminal device further comprising a distance output means for outputting a distance included in the one or more pieces of photographing object information, in contrast to the first.

  With this configuration, it is possible to output the distance to the object that is captured by the camera and displayed on the display.

  In addition, in the terminal device of the twenty-third invention, in contrast to the second one or the second two, the region output means indicates the object corresponding to the distance as the distance included in the one or more shooting object information is larger. It is a terminal device that outputs an area with a smaller size.

  With this configuration, it is possible to output the area of the object photographed by the camera and displayed on the display to the terminal device in accordance with the size of the object.

  In addition, the terminal device according to the twenty-fourth aspect of the present invention further comprises an object-related information receiving unit that receives object-related information from an external device with respect to any of the eighteenth to second-third. The terminal device further comprises object related information output means for outputting the object related information received by the object related information receiving unit.

  With this configuration, it is possible to output information related to the object that is captured by the camera and displayed on the display.

  Further, the terminal device of the twenty-fifth aspect of the present invention further comprises a first display means and a second display means, which are two separated display parts, with respect to the second, and the camera video output means is The video acquired by the camera unit is displayed on the first display unit, and the object-related information output unit is a terminal device that displays the object-related information received by the object-related information receiving unit on the second display unit.

  With this configuration, it is possible to output both video captured by the camera and information related to objects in the video to appropriate areas.

  Also, in the terminal device of the 26th invention, the area output means does not output the area information of other objects, but only the area information of one object, for any of the 18th to 25th Is superimposed on the video output by the output unit so that it can be visually distinguished from other area information.

  With this configuration, it is possible to appropriately output the object area in the video image captured by the camera.

  In addition, in the terminal device of the 27th invention, the object related information output means does not output the object related information of other objects to any one of the second to the sixth, but instead of the object related information of one object. It is a terminal device that outputs only object related information.

  With this configuration, it is possible to appropriately output information related to the object in the video being shot by the camera.

  Also, in the terminal device of the twenty-eighth aspect of the invention, in contrast to the twenty-sixth or the twenty-seventh aspect, the shooting object determining unit determines the object that is present at the closest position in the video shot by the camera unit. The shooting object or the shooting object of the object located closest to the center in the video shot by the camera unit or the shooting object of the object having the largest area in the video shot by the camera unit is determined. It is a terminal device.

  With this configuration, it is possible to output information or a region related to one object in a video shot by the camera.

  In addition, in the terminal device of the twenty-ninth invention, in response to any one of the eighteenth to the twenty-eighth, the terminal device receives a user's instruction, a first receiving unit, and a receiving unit receives the instruction The terminal device further includes a video output by the output unit and an output fixing unit that fixes the area.

  With such a configuration, since the video imaged by the camera can be fixed and fixed, the operation on the object in the video becomes easy.

  Further, in the terminal device of the thirtieth aspect, in contrast to the ninth aspect, the photographing object information includes an object identifier, and is accepted by a second accepting unit that accepts an input to a fixed area, and a second accepting unit. An object identifier acquisition unit that acquires an object identifier for an area corresponding to the input, and an object identifier transmission unit that transmits the object identifier acquired by the object identifier acquisition unit to one of the one or more web server devices, The object related information receiving unit is a terminal device that receives object related information in response to the transmission of the object identifier, and the transmission of the object identifier is performed according to an instruction from the outside.

  With such a configuration, since the video imaged by the camera can be fixed and fixed, the operation on the object in the video becomes easy.

  The terminal device according to the thirty-first aspect of the present invention provides a camera layer, which is a layer for outputting video acquired by the camera unit, or a layer for outputting an area corresponding to an object. Among the plurality of layers including the object identification layer, the web search layer that outputs the object related information, and the current layer storage unit that can store the current layer that is the current layer, and the input that sets the current layer And a current layer changing unit for changing the current layer of the current layer storage unit according to the input received by the third receiving unit, and when the current layer is a camera layer, the camera video output The means outputs the video acquired by the camera unit, and the current layer is the object identification layer. In this case, the camera video output means outputs the video acquired by the camera unit, and the area output means outputs the video image superimposed on the video output by the output part so that the area can be visually distinguished from other areas. When the current layer is a web search layer, the camera video output means outputs the video acquired by the camera section, and the area output means outputs the output section so that the area can be visually distinguished from other areas. The object related information output means is a terminal device that outputs the object related information.

  With such a configuration, it is possible to display appropriate information for an object in a video image taken by the camera.

  The terminal device according to the thirty-second invention includes an object information storage unit capable of storing one or more object information having an object identifier for identifying an object and area information indicating an area in which the object exists, and a terminal A terminal information acquisition unit for acquiring terminal information including position information indicating the position of the device and direction information indicating a direction in which the terminal device is facing; position information included in the terminal information acquired by the terminal information acquisition unit; and a terminal A shooting object determination unit that applies the direction information included in the information and the pre-stored viewing angle to the area information of each object information in the object information storage unit, and determines one or more objects being shot by the terminal device Among the one or more pieces of object information corresponding to each of the one or more objects determined by the shooting object determination unit A shooting object information acquisition unit that acquires one or more shooting object information including a bespoke from the object information storage unit, a camera unit that acquires a video, and a video acquired by the camera unit, and outputs one or more shooting object information Is a terminal device.

  With such a configuration, it is possible to display appropriate information for an object in a video image taken by the camera.

  According to the information system of the present invention, it is possible to recognize an object photographed by a camera and output information related to the object.

  Hereinafter, embodiments of an information system and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

  (Embodiment 1)

  In the present embodiment, an information system 1 that recognizes an object appearing in an image captured by a camera and outputs information related to the object will be described.

  Information related to the object includes the object area, the web page related to the object, the symbol image of the object, symbolic music and sound information, and guidance information (images, character strings, audio) Etc.).

  FIG. 1 is a conceptual diagram of an information system 1 in the present embodiment. The information system 1 includes a server device 11, one or more web server devices 12, and a terminal device 13. The terminal device 13 is a terminal having a two-screen display in FIG. 1, but may be a terminal having a one-screen display. The terminal device 13 has a camera function, and can output an area of an object (building, etc.) in the image being taken, information related to the object, etc. when shooting an external landscape, a building, or the like. . Moreover, the server apparatus 11, the web server apparatus 12, and the terminal device 13 are normally communicable.

  The information system 1 may be realized only by the server device 11 and the terminal device 13. In such a case, the server device 11 may have the function of a web server device. Further, the web server device 12 is a device that stores a so-called web page on the web, but the stored information may be other information such as advertisement information instead of the web page. The format and content of the information stored in the web server device 12 are not limited.

  FIG. 2 is a block diagram of the information system 1 in the present embodiment. The server device 11 includes an object information storage unit 111, a server reception unit 112, a shooting object determination unit 113, a distance calculation unit 114, a shooting object information acquisition unit 115, and a server transmission unit 116.

  The web server device 12 includes an object related information storage unit 121, an object related information acquisition unit 122, and an object related information transmission unit 123.

  The terminal device 13 includes a current layer storage unit 1301, a reception unit 1302, a terminal information acquisition unit 1303, a camera unit 1304, a transmission unit 1305, a reception unit 1306, an object related information reception unit 1307, an output information acquisition unit 1308, an output unit 1309, A current layer changing unit 1310, an output fixing unit 1311, an object identifier acquiring unit 1312, an object identifier transmitting unit 1313, and a display unit 1314 are provided.

  The receiving unit 1302 includes a first receiving unit 13021, a second receiving unit 13022, and a third receiving unit 13023. The terminal information acquisition unit 1303 includes position information acquisition means 13031 and direction information acquisition means 13032. The output unit 1309 includes camera video output means 13091, area output means 13092, distance output means 13093, and object related information output means 13094. The display unit 1314 includes first display means 13141 and second display means 13142.

  The object information storage unit 111 can store one or more pieces of object information. The object information is information having an object identifier and area information indicating an area where the object exists. An object is a geographic object that is usually an object that appears on a map or on a geography. The object is, for example, a building, a mountain, a river, a scenic spot, a park, or the like. The object may be, for example, a store or a company in a building. The content of the object is not limited as long as it appears on the map or the geography. The object information storage unit 111 may store map information. In such a case, the map information includes one or more object information. The object identifier is information for identifying an object, such as an object name, an ID for identifying the object, a URL of a web page in which information about the object is stored, a URI, and the like. The area information is usually information on a two-dimensional area. However, information of a three-dimensional region or one-dimensional information (point information) may be used. That is, the area information may be position information (X, Y) that specifies the position (point) of the object. This (X, Y) is, for example, (latitude, longitude), and is information indicating the center of gravity of the object region, for example. When the area is a rectangle, the area information is, for example, information on position information (longitude, latitude) of the upper left point and lower right point of the rectangle. Furthermore, the area information includes, for example, a set of points (eight points) constituting the vertex of a three-dimensional object solid figure (for example, hexahedron) or four points having the height of the solid figure (for example, hexahedron). And so on.

  The object information storage unit 111 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. The process in which the object information is stored in the object information storage unit 111 does not matter. For example, object information may be stored in the object information storage unit 111 via a recording medium, and object information transmitted via a communication line or the like is stored in the object information storage unit 111. Alternatively, the object information input via the input device may be stored in the object information storage unit 111.

  The server reception unit 112 receives terminal information from the terminal device 13. The terminal information includes position information indicating the position of the terminal device 13 and direction information indicating the direction in which the terminal device 13 is facing. The terminal information usually has a terminal identifier that identifies the terminal device 13. The position information is usually (latitude, longitude) information, but may be information indicating a position in space. The information indicating the position in the space is, for example, (latitude, longitude, altitude). The direction information may include direction information such as east, west, south, and north. The direction information may include the orientation of the terminal (upward, downward, etc.). The direction information is, for example, 56.2 ° (an orientation in which north is 0 ° and east is 90 °), south-southwest, and the like. The direction information may include, for example, an angle in the height direction. The direction information may be, for example, (Yaw, Pitch, Roll). “Yaw” of (Yaw, Pitch, Roll) is the angle with respect to the vertical axis and the degree of left and right shaking of the tip (length etc.), “Pitch” is the angle with respect to the horizontal axis and the degree of shaking of the tip up and down, “Roll” Is the angle with respect to the vertical axis and the degree of lateral shaking of the entire terminal device 13. The position information is acquired by, for example, a GPS receiver. The direction information is acquired by, for example, a compass or a compass and a gyro. There is no limitation on the means for acquiring the direction information and the position information. Further, the terminal information preferably includes a terminal identifier for identifying the terminal device 13 (for example, an IP address, a telephone number, an ID, a MAC address, etc. used for transmission / reception of information).

  The server receiving unit 112 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast.

  The shooting object determination unit 113 applies the position information included in the terminal information, the direction information included in the terminal information, and the viewing angle to the area information of each object information in the object information storage unit 111, and the terminal device 13 performs shooting. Determine one or more objects. The viewing angle is an angle at which the camera unit 1304 can shoot, and may be an angle on a plane or a three-dimensional angle including a height direction. The three-dimensional angle is obtained, for example, by a plane angle (θ1) and a height direction angle (θ2). The viewing angle may be stored in advance by the shooting object determination unit 113 or may be included in the terminal information. Here, “determining an object” may be to acquire an object identifier, to acquire all or part of object information, or to acquire a record ID of a table that manages object information It may be to do. In other words, “determining an object” only needs to acquire information for specifying an object.

  For example, the shooting object determination unit 113 may be the shooting object of the object that is closest to the terminal device 13 in the video shot by the camera unit 1304 or the center of the video shot by the camera unit 1304. A shooting object of an object located closer to the object or an object having the largest area (on a plane or a solid) in a video shot by the camera unit 1304 may be determined.

  The shooting object determination unit 113 determines the distance between the terminal device 13 and the object from the position information included in the terminal information and the object position information indicating the position of the object acquired from the area information of each object information in the object information storage unit 111. , And a predetermined number of objects may be determined as shooting objects from the distance within the predetermined distance or closer to the distance. In such a case, the shooting object acquisition algorithm is not accurate, but high-speed processing is possible. Needless to say, the distance calculation may be performed by the distance calculation unit 114.

  More specifically, the shooting object determination unit 113 preferably determines one or more objects being shot by the terminal device 13 using an algorithm as shown in a flowchart of FIG.

  The shooting object determination unit 113 can be usually realized by an MPU, a memory, or the like. The processing procedure of the shooting object determination unit 113 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The distance calculation unit 114 determines the distance between each object determined by the shooting object determination unit 113 and the terminal device 13 for each object determined by the shooting object determination unit 113 from the object information of each object and the position information included in the terminal information. To calculate. Here, the distance is the distance between the center of gravity of the area and the position indicated by the position information, the distance between any point in the area and the position indicated by the position information, and the point in the area closest to the position indicated by the position information. It may be distance. That is, there can be various distance calculation criteria. Further, the position information may be information generated from the area information. The position information is obtained, for example, by acquiring the center of gravity of the area obtained from the area information. The distance calculation unit 114 can usually be realized by an MPU, a memory, or the like. The processing procedure of the distance calculation unit 114 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The shooting object information acquisition unit 115 receives, from the object information storage unit 111, one or more shooting object information including area information among one or more object information corresponding to one or more objects determined by the shooting object determination unit 113. get. The shooting object information acquisition unit 115 may include any information as shooting object information. For example, the shooting object information acquisition unit 115 may include the distance calculated by the distance calculation unit 114 for each shooting object. The shooting object information acquisition unit 115 can usually be realized by an MPU, a memory, or the like. The processing procedure of the shooting object information acquisition unit 115 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The server transmission unit 116 transmits the shooting object information acquired by the shooting object information acquisition unit 115 to the terminal device 13. The server transmission unit 116 is usually realized by a wireless or wired communication unit, but may be realized by a broadcasting unit.

  The object related information storage unit 121 can store object related information that is information related to an object. The object-related information is a web page, in particular, an e-commerce or reservation web page, a symbol image of the object, or sound information (including music information) about the object, advertising information for advertising the object, and reaching the object. Any information related to the object can be used, such as route guidance information (sound, image, text, etc.), a game program that recalls the object. The object-related information storage unit 121 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

  The process in which object relevant information is memorize | stored in the object relevant information storage part 121 is not ask | required. For example, the object-related information may be stored in the object-related information storage unit 121 via a recording medium, and the object-related information transmitted via a communication line or the like is stored in the object-related information storage unit 121. Alternatively, the object related information input via the input device may be stored in the object related information storage unit 121.

  The object related information acquisition unit 122 acquires object related information in response to an instruction from the outside. The instruction from the outside is, for example, an instruction generated when the user points to an area (an object area, a frame, or the like) output to the terminal device 13. Such an instruction may arrive directly from the terminal device 13 or may arrive via the server device 11. The object related information may be a part of information constituting the object related information. The object related information acquisition unit 122 can be usually realized by an MPU, a memory, or the like. The processing procedure of the object related information acquisition unit 122 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The object related information transmission unit 123 transmits the object related information acquired by the object related information acquisition unit 122. The object-related information transmission unit 123 is usually realized by a wireless or wired communication unit, but may be realized by a broadcasting unit.

  The current layer storage unit 1301 can store the current layer. The current layer is the current layer. A layer is a hierarchy of processing performed by the terminal device 13. The terminal device 13 usually has a plurality of layers. The layers include, for example, a camera layer that is a layer that outputs video acquired by the camera unit 1304, an object identification layer that is a layer that outputs an area corresponding to an object, and a web search layer that is a layer that outputs object-related information. is there. For example, the terminal device 13 has layers of a camera layer, an object identification layer, and a web search layer from the bottom. For example, the terminal device 13 may have a layer indicating another application such as a navigation layer, a game layer, a music layer, or a photo layer. The navigation layer is a layer for outputting route guidance to an object. The game layer is a layer that outputs a game program and provides an environment where a game can be played. The music layer is a layer that outputs music. A photo layer is a layer that outputs one or more photos.

  The current layer storage unit 1301 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. The process of storing the current layer in the current layer storage unit 1301 does not matter. For example, the current layer may be stored in the current layer storage unit 1301 via a recording medium, and the current layer transmitted via a communication line or the like is stored in the current layer storage unit 1301. Alternatively, the current layer input via the input device may be stored in the current layer storage unit 1301.

  The accepting unit 1302 accepts inputs such as instructions and data from the user. Any means such as a keyboard, a mouse, a numeric keypad, or a menu screen may be used for inputting instructions and data. The accepting unit 1302 can be realized by a device driver for input means such as a keyboard, control software for a menu screen, or the like.

  The first receiving unit 13021 constituting the receiving unit 1302 receives a user instruction. The second accepting unit 13022 accepts an input for the fixed area. Further, the third receiving unit 13023 receives an input for setting the current layer. Input means such as user instructions, input to a fixed area, and input for setting the current layer may be the same means or different means.

  The terminal information acquisition unit 1303 acquires terminal information including position information indicating the position of the terminal device 13 and direction information indicating the direction in which the terminal device 13 is facing.

  Position information acquisition means 13031 constituting the terminal information acquisition unit 1303 acquires position information indicating the position of the terminal device 13. The position information acquisition unit 13031 is configured by, for example, a GPS receiver. The position information acquisition unit 13031 is, for example, a radio wave reception unit and a processing unit (for example, an MPU or a memory) that acquires position information from the intensity of three or more radio waves received from three or more base stations. It is also possible to configure the device. The method by which the position information acquisition unit 13031 acquires position information does not matter. The position information may be a two-dimensional position or a position in a three-dimensional space.

  The direction information acquisition unit 13032 acquires direction information indicating the direction in which the terminal device 13 is facing. The direction information may be information indicating a two-dimensional direction or information indicating a three-dimensional direction (including the vertical direction). The direction information acquisition unit 13032 is realized by a compass, for example. In addition, the direction information acquisition unit 13032 is realized by, for example, a compass and a gyro.

  Further, the terminal information may include the viewing angle of the camera unit 1304. The viewing angle is synonymous with viewing angle information. In such a case, for example, it is assumed that the terminal information acquisition unit 1303 stores the viewing angle of the camera unit 1304 in advance. The viewing angle may be a two-dimensional angle or a three-dimensional angle. In addition, if the position, direction, and viewing angle of the terminal device 13 (camera unit 1304) are known, it is possible to know a region that can be photographed. That is, the terminal information is information for specifying an area to be photographed by the camera unit 1304. The area to be imaged is preferably a three-dimensional area, but may be a two-dimensional area.

  The terminal information may include a terminal identifier. The viewing angle and the terminal identifier are stored in a storage medium (not shown) included in the terminal information acquisition unit 1303, for example. The terminal identifier is information for identifying a terminal, such as a telephone number, an IP address, or a MAC address.

  The camera unit 1304 acquires a video. The camera unit 1304 is a so-called camera. The camera unit 1304 normally acquires an image currently displayed in the viewfinder. The camera unit 1304 can be realized by, for example, a CCD camera.

  The transmission unit 1305 transmits the terminal information acquired by the terminal information acquisition unit 1303 to the server device 11. The transmission unit 1305 is usually realized by wireless or wired communication means, but may be realized by broadcasting means.

  The receiving unit 1306 receives shooting object information in response to transmission of terminal information. The shooting object information usually includes an object identifier. The shooting object information may include a distance from the terminal device 13 to the object. The receiving unit 1306 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast.

  The object related information receiving unit 1307 receives object related information. The object related information receiving unit 1307 may receive the object related information in response to the transmission of the object identifier. The object related information receiving unit 1307 receives object related information from the web server device 12 or the server device 11. The object related information receiving unit 1307 may receive object related information from the web server device 12 via the server device 11. The object-related information receiving unit 1307 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast.

  The output information acquisition unit 1308 acquires output information including one or more area information included in the shooting object information received by the reception unit 1306. In general, the output information acquisition unit 1308 acquires output information having a pair of region information and an object identifier. The output information acquisition unit 1308 may acquire output information including one or more area information and one or more distances included in the shooting object information received by the reception unit 1306. Note that the output information acquisition unit 1308 does not perform any processing when simply passing the shooting object information to another element such as the output unit 1309. In such a case, the output information acquisition unit 1308 is not necessary. The output information acquisition unit 1308 may or may not include an output device such as a display or a speaker. The output information acquisition unit 1308 can be implemented by output device driver software, or output device driver software and an output device.

  The output unit 1309 outputs one or more pieces of information among video, audio, character strings, images, and the like. The output unit 1309 may be considered as including or not including an output device such as a display or a speaker. The output unit 1309 may be realized by output device driver software, or output device driver software and an output device.

  The camera video output unit 13091 outputs the video acquired by the camera unit 1304. The camera video output unit 13091 preferably displays the video acquired by the camera unit 1304 on the first display unit 13141.

  The area output unit 13092 outputs area information included in one or more pieces of output information acquired by the output information acquisition unit 1308 so as to overlap the video output by the camera video output unit 13091. In such a case, region information (for example, a frame) is output so that the region of the object indicated by the region information can be visually determined. The area output means 13092 is a camera video output means so that an area having a size corresponding to the area information included in the one or more pieces of output information acquired by the output information acquisition unit 1308 can be visually distinguished from other areas. The image is output by being overlaid on the video output by 13091. It is preferable that the area output unit 13092 outputs an object corresponding to the distance with a smaller size as the distance included in the one or more pieces of output information acquired by the output information acquisition unit 1308 is larger. The area output means 13092 does not output the area of the other object, and superimposes only the area of one object on the video output by the camera video output means 13091 so that it can be visually distinguished from the other areas. It is preferable to output. “Only one object area” is intended not to output other areas, and is not intended to output other types of information (information other than area information). In the above, the area information included in the one or more pieces of output information acquired by the output information acquisition unit 1308 may be the area information included in one or more shooting objects.

  The region output means 13092 may or may not include an output device such as a display. The area output means 13092 can be realized by output device driver software, or output device driver software and an output device.

  The distance output unit 13093 outputs a distance included in one or more shooting objects. The distance output unit 13093 is preferably output in association with the region output by the region output unit 13092. In addition, it is preferable that the distance output unit 13093 outputs the distance so as to be superimposed on the video output by the output unit 1309. Here, “corresponding to the region” is preferably output of the distance within the region, but may be output of the distance outside the region. The distance output unit 13093 may or may not include an output device such as a display or a speaker. The distance output means 13093 can be implemented by output device driver software, or output device driver software and an output device.

  The object related information output unit 13094 outputs the object related information received by the object related information receiving unit 1307.

  It is preferable that the object related information output unit 13094 displays the object related information received by the object related information receiving unit 1307 on the second display unit 13142. In such a case, the video acquired by the camera unit 1304 is displayed on the first display unit 13141, and the object related information is displayed on the second display unit 13142. The object related information output means 13094 preferably outputs only the object related information of one object without outputting the object related information of other objects. Here, “output only the object-related information of one object” means that the object-related information of another object is not output, and does not mean that other types of information are not output. The object related information output unit 13094 may be considered as including or not including an output device such as a display or a speaker. The object-related information output unit 13094 can be realized by output device driver software, or output device driver software and output device.

  The current layer changing unit 1310 changes the current layer in the current layer storage unit 1301 according to the input received by the third receiving unit 13023. It is assumed that the input received by the third receiving unit 13023 includes information for specifying the current layer (current layer or information for advancing the current layer (when the layers are in order)). The current layer changing unit 1310 can usually be realized by an MPU, a memory, or the like. The processing procedure of the current layer changing unit 1310 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  When the receiving unit 1302 receives an instruction, the output fixing unit 1311 fixes the video and the area that the output unit 1309 is outputting. Here, “fixed” refers to Pause and Resume. In other words, “fixed” means that even if the angle of the camera is moved, the image or area information output on the screen does not move. Then, the output fixing unit 1311 records the video and area being output by the output unit 1309 at least temporarily on a storage medium (not shown). Note that a technique for fixing an image being output is the same as a function of a so-called video movie, and is a known technique. The output fixing unit 1311 can be usually realized by an MPU, a memory, or the like. The processing procedure of the output fixing unit 1311 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The object identifier acquisition unit 1312 acquires an object identifier for an area corresponding to the input received by the second receiving unit 13022. The object identifier acquisition unit 1312 acquires, for example, an object identifier that is paired with the area information. For example, the object identifier acquisition unit 1312 acquires area information specified by the designated frame (area) from one or more pieces of shooting object information, and acquires an object identifier paired with the area information. Here, the object identifier may be any information that identifies an object, and may be, for example, a URL or an object name. In addition, the object identifier acquired by the object identifier acquisition unit 1312 and the object identifier processed by another component may be different types of information (for example, object ID and URL). The object identifier acquisition unit 1312 can usually be realized by an MPU, a memory, or the like. The processing procedure of the object identifier acquisition unit 1312 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The object identifier transmission unit 1313 transmits the object identifier acquired by the object identifier acquisition unit 1312 to one of the one or more web server devices 12. The object identifier transmission unit 1313 may transmit the object identifier to the server device 11. The object identifier transmission unit 1313 may transmit an object-related information transmission instruction to the web server device 12 that can be specified by the object identifier. In such a case, it is also assumed that the object identifier is transmitted. The object identifier transmission unit 1313 is usually realized by a wireless or wired communication unit, but may be realized by a broadcasting unit.

  The display unit 1314 is a device that displays information. The first display unit 13141 is a first device that displays information. The second display unit 13142 is a second device that displays information. For example, the first display unit 13141 and the second display unit 13142 are preferably physically separated, but each display may be divided into two windows.

  Next, the operation of the information system 1 will be described. First, operation | movement of the server apparatus 11 is demonstrated using the flowchart of FIG.

  (Step S <b> 401) The server reception unit 112 determines whether terminal information has been received from the terminal device 13. If terminal information is received, it will go to step S402, and if terminal information is not received, it will return to step S401.

  (Step S402) Using the terminal information received in step S401, the shooting object determination unit 113 uses the location information and / or direction information of the terminal information as region information of each object information in the object information storage unit 111. Apply one or more objects that are captured by the terminal device 13 (shown on the display of the terminal device 13). Such a process is called a shooting object determination process. Note that it is preferable that the shooting object determination unit 113 determines one or more objects being shot by the terminal device 13 also using a pre-stored viewing angle. Note that an object being photographed by the terminal device 13 is also referred to as a photographing object. The shooting object determination process will be described with reference to the flowchart of FIG.

  (Step S403) The distance calculation unit 114 acquires position information included in the terminal information received in Step S401. Such position information is information indicating the position of the terminal device 13.

  (Step S404) The distance calculation unit 114 substitutes 1 for a counter i.

  (Step S405) The distance calculation unit 114 determines whether or not the i-th shooting object exists. If the i-th shooting object exists, the process goes to step S406, and if the i-th shooting object does not exist, the process goes to step S410.

  (Step S406) The distance calculation unit 114 acquires the position information of the i-th shooting object from the object information storage unit 111. Note that the position information of the i-th shooting object is included in the object information of the i-th shooting object.

  (Step S407) The distance calculation unit 114 calculates the distance between the point indicated by the position information acquired in step S403 and the point indicated by the position information acquired in step S406. A technique for calculating the distance between the two points from each position information of the two points is a known technique. The distance calculated here is the distance between the terminal device 13 and the shooting object.

  (Step S408) The shooting object information acquisition unit 115 acquires information including area information of the i-th shooting object from the object information storage unit 111. The shooting object information acquisition unit 115 configures shooting object information having the i-th shooting object information acquired from the object information storage unit 111 and the distance acquired in step S407. Next, the shooting object information acquisition unit 115 temporarily stores the i-th shooting object information in the buffer.

  (Step S409) The distance calculation unit 114 increments the counter i by 1. The process returns to step S405.

  (Step S <b> 410) The server transmission unit 116 transmits one or more pieces of shooting object information temporarily stored in the buffer in step S <b> 408 to the terminal device 13. The process returns to step S401.

  In the flowchart of FIG. 4, the process ends when the power is turned off or the process ends.

  In the flowchart of FIG. 4, it is needless to say that the distance between the terminal device 13 and the shooting object may be calculated only once. That is, when the distance between the terminal device 13 and the shooting object is calculated and stored in step S402, the processing of steps S406 to S407 is not necessary.

  Next, the shooting object determination process in step S402 will be described with reference to the flowchart of FIG.

  (Step S501) The shooting object determination unit 113 acquires position information included in the terminal information received in step S401.

  (Step S502) The shooting object determination unit 113 acquires an angle at which shooting is not performed using the direction information included in the terminal information received in step S401 and the viewing angle. Then, the shooting object determination unit 113 registers (records) the angle (the angle of the invisible region) on the recording medium. For example, when the direction information is 90 degrees (for example, true east) and the viewing angle is 40 degrees, the captured angle is “70 degrees to 110 degrees”. Therefore, the photographing object determination unit 113 registers “0 <= invisible region angle <70” and “110 <invisible region angle <= 360”. Here, the shooting object determination unit 113 may acquire and register the shooting angle (the angle of the visible region). That is, it is assumed that the registration of the angle of the visible region is equivalent to the registration of the angle of the invisible region.

  (Step S503) The shooting object determination unit 113 sorts the object information in the object information storage unit 111 in the order of objects closer to the terminal device 13, using the position information acquired in step S501. Such processing is called object sort processing. The object sort process will be described with reference to the flowchart of FIG.

  (Step S504) The shooting object determination unit 113 substitutes 1 for the counter i.

  (Step S505) The shooting object determination unit 113 determines whether or not the i-th object exists in the object information storage unit 111. If the i-th object exists, the process proceeds to step S506, and if the i-th object does not exist, the process returns to the upper process.

  (Step S506) The shooting object determining unit 113 acquires area information included in the object information of the i-th object from the object information storage unit 111.

  (Step S507) The shooting object determination unit 113 uses the position information acquired in step S501 and the region information acquired in step S505 to display the i-th object on the shooting object (display of the terminal device 13). ) Or not. Such a determination is referred to as a visual determination process. The visibility determination process will be described with reference to the flowcharts of FIGS.

  (Step S508) As a result of the determination in step S506, the shooting object determination unit 113 goes to step S509 if the i-th object is a shooting object, and goes to step S510 if it is not a shooting object.

  (Step S509) The shooting object determination unit 113 stores at least the object identifier of the i-th object in a storage medium (not shown).

  (Step S510) The photographing object information determination unit 113 determines whether or not a visible area exists from the registered invisible area or visible area. If the visible area exists, the process goes to step S511, and if not, the process returns to the upper process. Such determination may be performed in a two-dimensional plane or may be performed in a three-dimensional space.

  (Step S511) The photographing object determining unit 113 increments the counter i by 1. The process returns to step S505.

  In the flowchart of FIG. 5, the angle of the invisible region is registered. However, the angle of the visible region may be registered, and only an object that falls within the angle may be processed thereafter.

  Further, in step S510 of the flowchart of FIG. 5, the shooting object information determination unit 113 is arranged in the order of the short distance from the terminal device 13 (the order may be the closest in the set of objects in which the range and angle are divided). When an object is inspected and information on an invisible area is registered as an invisible area as a result of shooting the object previously determined that the terminal device 13 is shooting, the visible area disappears (all become invisible areas). In this case, it is explained that the inspection of whether or not the object is photographed on the terminal device 13 is terminated.

Furthermore, specifically, the process of step S510 is as follows. In other words, the shooting object information determination unit 113 inspects the objects in the order from the terminal device 13, and information on an area that becomes invisible by shooting of the object previously determined that the terminal device 13 is shooting (here, normally, A minimum angle (θ ₁ ) and a maximum angle (θ ₂ ), which will be described later, are registered as an invisible region, information on a region that becomes an invisible region is acquired by the object to be inspected next and the terminal device 13, and the minimum angle (θ ₃ ) It is determined whether all the areas specified by the maximum angle (θ ₄ ) have already been registered as invisible areas, and if not registered as invisible areas, the terminal device 13 determines the object to be inspected next. The area determined by the minimum angle (θ ₃ ) and the maximum angle (θ ₄ ) is registered as an invisible area. This process is repeated, and the inspection of the object is terminated when all the viewing angles of the terminal information are invisible.

Further, in the flowchart of FIG. 5, the process of step S510 may be omitted.
Furthermore, in the flowchart of FIG. 5, when the visible region no longer exists, the detection processing of the object being photographed is terminated. However, the object detection process may be terminated depending on other conditions, such as when an object in the visible region corresponding to the threshold number is found, or an object that matches a predetermined condition is found.

  Next, the object sort processing in step S502 will be described using the flowchart of FIG.

  (Step S601) The photographic object determination unit 113 substitutes 1 for a counter i.

  (Step S602) The shooting object determination unit 113 determines whether or not the i-th object exists in the object information storage unit 111. If the i-th object exists, the process goes to step S603, and if the i-th object does not exist, the process goes to step S607.

  (Step S <b> 603) The shooting object determination unit 113 acquires the position information of the i-th object from the object information storage unit 111. Note that the position information of the i-th object is included in the object information of the i-th object.

  (Step S604) The shooting object determining unit 113 calculates the distance between the point indicated by the position information acquired in step S403 and the point indicated by the position information acquired in step S603.

  (Step S605) The shooting object determination unit 113 pairs the distance acquired in Step S604 and the object information (or part of the information) of the i-th object and temporarily stores them in a recording medium (not shown). .

  (Step S606) The photographing object determining unit 113 increments the counter i by one. The process returns to step S602.

  (Step S607) The shooting object determination unit 113 sorts the object information in the order of closer (smaller) distance using the distance as a key. Return to upper process.

  Next, the visibility determination process in step S506 will be described with reference to the flowchart in FIG. The visibility determination process here is a first visibility determination process. The first visibility determination process is a process for determining whether or not the object is output on the display of the terminal device 13 using only the planar information of the object.

  (Step S701) The shooting object determining unit 113 substitutes 1 for the counter i.

  (Step S702) The shooting object determination unit 113 determines whether or not the i-th point exists in the area information of the object to be determined. If the i-th point exists, the process goes to step S703, and if the i-th point does not exist, the process goes to step S705. The region information is assumed to be composed of points indicating the outer shape, for example. The points indicating the outer shape include four points if the region indicated by the region information is a tetragon, and eight points if the region is an octagon. The point is usually indicated by (latitude, longitude) or (x coordinate, y coordinate).

  (Step S703) The shooting object determining unit 113 calculates an angle (θ) between the i-th point and the position information, and temporarily stores the angle (θ) in a buffer. The reference for the angle (θ) is, for example, true north. However, the standard only needs to be constant and is not necessarily true north. The position information is information indicating the position of the terminal device 13.

  (Step S704) The photographing object determining unit 113 increments the counter i by 1. The process returns to step S702.

(Step S705) The shooting object determining unit 113 acquires the minimum angle (θ ₁ ) and the maximum angle (θ ₂ ) from the angle (θ) temporarily stored in the buffer in step S703.

(Step S706) The shooting object determining unit 113 determines whether or not all the areas of the minimum angle (θ ₁ ) and the maximum angle (θ ₂ ) acquired in step S705 are already registered angle areas. To do. If it is an area of an already registered angle, go to step S710, and if it does not include any of the already registered angle areas, go to step S707.

  (Step S707) The shooting object determination unit 113 determines that this object is a shooting object. Information (for example, “1”) indicating that the object is a shooting object is substituted as a return value.

(Step S708) Since the shooting object determining unit 113 determines that this object is a shooting object, in addition to the angle currently registered as the invisible region, the shooting object determination unit 113 sets the minimum angle (θ ₁ ) and the maximum angle (θ ₂ ). Acquires the width (region) of the angle that makes the region an invisible region. For example, if “30 to 40 degrees” is currently registered, the minimum angle (θ ₁ ) is 35 degrees, and the maximum angle (θ ₂ ) is 45 degrees, the shooting object determination unit 113 Acquires “30 degrees to 45 degrees”. Also, for example, when “30 to 40 degrees” is currently registered, the minimum angle (θ ₁ ) is 55 degrees, and the maximum angle (θ ₂ ) is 60 degrees, “55 to 60 degrees”. Get "degree".

  (Step S709) The shooting object determining unit 113 registers the angle acquired in step S708. For example, in the above, “30 degrees to 45 degrees” is overwritten with the information “30 degrees to 40 degrees”. For example, in the above, “55 degrees to 60 degrees” is additionally written. Return to upper process. At this time, a return value is also returned.

  (Step S710) The shooting object determination unit 113 determines that the shooting object determination unit 113 is a non-shooting object. Information indicating that the object is a non-photographing object (for example, “0”) is substituted as a return value. Return to upper process. At this time, a return value is also returned.

  Next, the visibility determination process in step S506 will be described with reference to the flowchart in FIG. The visibility determination process here is a second visibility determination process. The second visibility determination process is a process for determining whether or not the object is output on the display of the terminal device 13 using only the three-dimensional information of the object. In the flowchart of FIG. 8, the description of the same steps as those of the flowchart of FIG.

(Step S801) The shooting object determination unit 113 calculates the angle (θ _h ) in the height direction between the i-th point (x1, y1, z1) and the position information (x0, y0, z0), and stores it in the buffer. To do.

(Step S802) The shooting object determining unit 113 acquires the angle (θ ₃ ) in the height direction from the viewpoint (position information). θ ₃ is the maximum angle among θ _h stored in step S801.

(Step S803) The photographing object determining unit 113 determines whether or not an angle equal to or higher than the angle (θ ₃ ) in the height direction has already been registered in the region of the minimum angle (θ ₁ ) and the maximum angle (θ ₂ ). Judging. If it is registered, the process goes to step S710, and if not registered, the process goes to step S707. If an angle equal to or higher than the angle (θ ₃ ) in the height direction has already been registered, it indicates that the target object is in the invisible region and is not a shooting object.

(Step S804) Since the shooting object determination unit 113 determines that this object is a shooting object, a plane having a minimum angle (θ ₁ ) and a maximum angle (θ ₂ ) in addition to the angle currently registered as the invisible region. Acquires the width (region) of the angle that makes the region an invisible region. For example, if “30 to 40 degrees” is currently registered, the minimum angle (θ ₁ ) is 35 degrees, and the maximum angle (θ ₂ ) is 45 degrees, the shooting object determination unit 113 Acquires “30 degrees to 45 degrees”.

(Step S805) The shooting object determination unit 113 acquires the angle (θ ₃ ) in the height direction because this object is a shooting object. For example, when the height angle “30 degrees” is registered in the plane area of the minimum angle (θ ₁ ) and the maximum angle (θ ₂ ), the acquired angle in the height direction (θ ₃ ) Is “35 degrees”, “35 degrees” is acquired.

(Step S806) The shooting object determining unit 113 associates the angle of the planar area (for example, “30 degrees to 45 degrees”) acquired in Step S804 with the angle in the height direction (for example, “35 degrees”). Accumulate (register). Return to upper process. At this time, a return value is also returned. Note that the shooting object determination unit 113 determines that the angle in the height direction currently registered as the invisible area in the plane area of the minimum angle (θ ₁ ) and the maximum angle (θ ₂ ) is the angle in the height direction (θ ₃₎ determines the whether or, if the height direction of the angle (theta ₃₎ above, do not accumulate angle (theta ₃₎ in the height direction.

  Next, the operation of the web server device 12 of the information system 1 will be described. The object related information acquisition unit 122 of the web server device 12 acquires object related information from the object related information storage unit 121 in response to an instruction from the outside (usually the terminal device 13 or the server device 11). Then, the object related information transmission unit 123 transmits the object related information acquired by the object related information acquisition unit 122 to the terminal device 13 or the server device 11.

  Next, operation | movement of the terminal device 13 is demonstrated using the flowchart of FIG.

  (Step S901) The receiving unit 1302 determines whether an instruction or the like has been received. If an instruction or the like is accepted, the process goes to step S902. If an instruction or the like is not accepted, the process returns to step S901.

  (Step S902) First receiving means 13021 determines whether or not the instruction received in step S901 is an object acquisition instruction. The object acquisition instruction is an instruction to acquire information regarding the object being photographed. If it is an object acquisition instruction, the process proceeds to step S903, and if it is not an object acquisition instruction, the process proceeds to step S909.

  (Step S903) The position information acquisition unit 13031 of the terminal information acquisition unit 1303 acquires the position information of the terminal device 13.

  (Step S904) The direction information acquisition unit 13032 acquires direction information indicating the direction in which the terminal device 13 is facing.

  (Step S905) The terminal information acquisition unit 1303 configures terminal information including the position information acquired in Step S903 and the direction information acquired in Step S904. In addition, it is suitable for the terminal information acquisition part 1303 to comprise the terminal information containing the viewing angle stored beforehand.

  (Step S906) The transmission unit 1305 transmits the terminal information configured in Step S905 to the server device 11.

  (Step S907) The receiving unit 1306 determines whether or not the shooting object information has been received in response to the transmission of the terminal information. If the shooting object information is received, the process proceeds to step S908. If the shooting object information is not received, the process returns to step S907.

  (Step S908) The output unit 1309 and the like perform processing for outputting the shooting object information received in step S907. Such processing is called shooting object information output processing. The process returns to step S901.

  (Step S909) The second accepting means 13022 judges whether or not the input accepted in Step S901 is an input for the area. The input to the area is an instruction to display an area (frame) indicating the object. If the input is for an area, the process goes to step S910, and if not, the process goes to step S911.

  (Step S910) The output unit 1309 and the like perform a process of outputting information related to the object for the designated area. Such processing is called object-related information output processing. The process returns to step S901.

  (Step S911) The third receiving unit 13023 determines whether or not the input received in step S901 is an input for setting the current layer. If the input is for setting the current layer, the process goes to step S912. If the input is not for setting the current layer, the process goes to step S913.

  (Step S912) The current layer changing unit 1310 changes the current layer of the current layer storage unit 1301 according to the input received in step S911.

  (Step S913) Each component performs an operation corresponding to the received instruction. For example, if the received instruction is a shooting start instruction, the camera unit 1304 is activated to acquire a video, and the camera video output unit 13091 outputs the video acquired by the camera unit 1304. Further, for example, when the received instruction is an output fixing instruction, the output fixing unit 1311 fixes the video and the region being output by the output unit 1309.

  Next, the shooting object information output process in step S908 will be described with reference to the flowchart of FIG.

  (Step S1001) The output unit 1309 reads the current layer from the current layer storage unit 1301.

  (Step S1002) The output information acquisition unit 1308 substitutes 1 for the counter i.

  (Step S1003) The output information acquisition unit 1308 determines whether or not the i-th shooting object exists in the shooting object information to be output. If the i-th shooting object exists, the process proceeds to step S1004. If the i-th shooting object does not exist, the process returns to the upper process.

  (Step S1004) The output information acquisition unit 1308 acquires area information of the i-th shooting object.

  (Step S1005) The output unit 1309 determines the frame size from the area information of the i-th shooting object. The frame size is usually a frame (usually a rectangle) having a size corresponding to the area information.

  (Step S1006) The output unit 1309 determines whether or not the current layer read in step S1001 is an object identification layer. If it is the object identification layer, go to step S1007, and if it is not the object identification layer, go to step S1010.

  (Step S1007) The region output means 13092 configures a frame that matches the frame size determined in step S1005. The frame is, for example, a rectangular area window, object, figure, or the like.

  (Step S1008) The area output unit 13092 outputs the frame configured in step S1007 to the area indicated by the area information. The region output unit 13092 normally outputs a frame in a manner that overlaps the video output by the camera video output unit 13091. This frame can also be said to be area information included in one or more pieces of output information acquired by the output information acquisition unit 1308.

  In this step, it is preferable that the distance output unit 13093 also outputs a distance included in one or more pieces of output information acquired by the output information acquisition unit 1308. It is more preferable to output the distance in association with the frame or the frame.

  (Step S1009) The output information acquisition unit 1308 increments the counter i by one. The process returns to step S1003.

  (Step S1010) The output unit 1309 determines whether or not the current layer read in step S1001 is a web search layer. If it is the web search layer, go to step S1007, and if it is not the web search layer, go to step S1009.

  (Step S1011) The output unit 1309 performs an object related information output process. Go to step S1009.

  In the flowchart of FIG. 10, it is not essential to output area information (for example, a frame). Moreover, the output of distance is not essential. The output unit 1309 may output any information of the shooting object information.

  Next, the object related information output processing in step S101 will be described with reference to the flowchart of FIG.

  (Step S1101) The object identifier acquisition unit 1312 acquires an object identifier for an area corresponding to the input received by the second receiving unit 13022.

  (Step S1102) The object identifier transmission unit 1313 transmits the object identifier acquired in step S1101 to one of the one or more web server devices 12. When the object identifier is the URL of the web server device 12, the object identifier transmission unit 1313 transmits the object identifier to the web server device 12 corresponding to the URL. The transmission of the object identifier includes transmission of some information for obtaining the object related information.

  (Step S1103) The object-related information receiving unit 1307 determines whether or not object-related information has been received in response to the transmission of the object identifier in step S1102. If the object related information is received, the process proceeds to step S1104. If the object related information is not received, the process returns to step S1103.

  (Step S1104) The output unit 1309 determines whether or not the object related information received in step S1103 is symbol information. If it is symbol information, go to step S1105, and if it is not symbol information, go to step S1106. Note that the symbol information is an image or icon that identifies an object. The symbol information is, for example, a symbol image or symbol music (particularly an icon) described later.

  (Step S1105) The object related information output unit 13094 reduces or enlarges the object related information received by the object related information receiving unit 1307 in accordance with the area.

  (Step S1106) The object related information output means 13094 outputs object related information. Return to upper process.

  Hereinafter, a specific operation of the information system 1 in the present embodiment will be described. A conceptual diagram of the information system 1 is shown in FIG.

  First, the technical concept of the information system 1 will be described. The information system 1 has a concept as shown in FIG. 12, for example. The information system 1 can realize Augmented Web Browsing. That is, the information system 1 superimposes information related to an object included in an image captured by the camera (a frame indicating a region, a web page, a symbol image, etc.) on the object or another region (another window or another). Display).

  Moreover, the terminal device 13 of the information system 1 has a plurality of layers as shown in FIG. A layer is a hierarchy of processing performed by the terminal device 13. The layers are, for example, a camera layer, a map sensing layer, an object identification layer, a web search API layer, a touch control layer. ) Etc. Such a structure of a plurality of layers is referred to as a Layer-based Media Integration Platform.

  The map sensing layer is a process layer for detecting map information. The touch control layer is a layer that accepts an instruction (usually touch) from the user and shifts to processing corresponding to the instruction. In addition, the camera layer, the object identification layer, and the web search layer are layers that can distinguish output information and perform processing. The map sensing layer and the touch control layer are layers for performing processing.

  Here, when the current layer is a camera layer, only the video acquired by the camera unit 1304 is output. When the current layer is an object identification layer, the video acquired by the camera unit 1304 and information (for example, a frame) about the object included in the video are output. Furthermore, when the current layer is a web search layer, the video acquired by the camera unit 1304, information about objects included in the video (for example, a frame), and information acquired from the web server device 12 are output. If the current layer is a web search layer, only the video acquired by the camera unit 1304 and information acquired from the web server device 12 may be output.

  Further, the information system 1 may have various layers in addition to the above-described layers (see FIG. 13). In addition to the layers described above, the various layers include, for example, a media superimposing layer and a navigation layer. The media superposition layer is a layer that superimposes and outputs various media. The navigation layer is a layer for navigating the way to the object.

  Next, the information system 1 will be described using two specific examples. Specific example 1 is a mode in which an object included in a captured video is recognized in consideration of only planar information of the object without considering the height of the object, and information related to the object is output. is there. Specific example 2 is an aspect in which the height of the object is also taken into account, the object included in the video being captured is recognized, and information related to the object is output.

  (Specific example 1)

  Now, the object information storage unit 111 of the server apparatus 11 holds the object information management table shown in FIG. The object information management table is a table that manages object information on the map. The object information management table manages one or more records having attribute values such as “object identifier”, “object name”, “URL”, “location information”, and “region information”. Here, the “object identifier” is an ID of an object. “Object name” indicates the name of the object. The URL is information indicating a location where a web page (object-related information here) describing the object exists. “Position information” is information for specifying the position of an object, and here is (latitude, longitude). “Area information” is information indicating a two-dimensional area in which an object is arranged, and here indicates three or more vertices of an object that is a polygon. The vertex is also (latitude, longitude) here.

  Moreover, the object relevant information storage part 121 of the web server apparatus 12 has the object relevant information shown in FIG. 15, for example. Here, the object-related information includes a symbol image, symbol music, and a web page. The symbol image is an image (icon) that symbolizes an object. The symbol music is music that symbolizes an object and an image (icon) indicating the music. The symbol music may be only music that symbolizes an object. A web page is a type of explanation information that describes an object. The web page may be a page that simply describes the object, may be advertisement information of the object, or may be a page that generates some action on the object (for example, order or reservation of goods or services). That is, the explanation information may be information that realizes processing that can be performed using a web page. In FIG. 15, the web page may include a symbol image, symbol music, and the like.

  Further, it is assumed that the terminal device 13 is a terminal having two displays. The terminal device 13 may be a terminal having one display or a terminal having three or more displays.

  Furthermore, it is assumed that the current layer storage unit 1301 of the terminal device 13 stores “current layer = camera layer”.

  In this case, it is assumed that the user has activated the terminal device 13. Then, the accepting unit 1302 accepts an activation instruction. Then, the camera unit 1304 reads “current layer = camera layer” from the current layer storage unit 1301. Then, the camera unit 1304 activates the camera. Then, as shown in FIG. 16, an external image is output to the first display unit 13141 of the terminal device 13.

  Next, it is assumed that the user has input an instruction to make the current layer an object identification layer. Then, the third receiving unit 13023 determines that the received input is an input for setting the current layer. Next, the current layer changing unit 1310 changes the current layer of the current layer storage unit 1301 to the object identification layer according to the received input.

  Next, the user inputs an object acquisition instruction to obtain information on the object in the video from the state of the terminal device 13 in FIG. The object acquisition instruction is preferably input by touching, but may be input by selection from a menu, input by an input means such as a keyboard, or the like. Next, the first receiving unit 13021 receives an object acquisition instruction.

  Next, position information acquisition means 13031 (here, for example, a GPS receiver) of the terminal information acquisition unit 1303 acquires the position information (X, Y) of the terminal device 13.

  Next, the direction information acquisition unit 13032 acquires direction information (θA) indicating the direction in which the terminal device 13 is facing. The direction information (θA) indicates, for example, a clockwise angle from true north (90 degrees for true east).

  In addition, the terminal information acquisition unit 1303 reads information on the viewing angle of the camera unit 1304 (simply referred to as “viewing angle”). Here, it is assumed that the viewing angle is “θa”. The camera unit 1304 stores a viewing angle “θa” in advance. And

  Next, the terminal information acquisition unit 1303 obtains terminal information (X, Y, θA, θa) having the acquired position information (X, Y), the acquired direction information (θA), and the viewing angle (θa). Configure.

  Next, the transmission unit 1305 transmits the configured terminal information (X, Y, θA, θa) to the server device 11.

  Then, the server reception unit 112 receives terminal information (X, Y, θA, θa) from the terminal device 13.

  Next, the server reception unit 112 determines a shooting object as follows. That is, the shooting object determination unit 113 acquires position information (X, Y) included in the received terminal information.

  Next, the shooting object determination unit 113 calculates an angle that becomes an invisible region by using the direction information (θA) and the viewing angle (θa) included in the terminal information. The shooting object determination unit 113 determines “(θA− (θa / 2)) <= visible region <= (θA + (θa / 2))” as an angle that becomes a visible region. Then, the angle excluding the visible region is calculated and stored as the invisible region angle. This concept is shown in FIG. When the visible region does not cross the true north line, the invisible region angle is “0 <= invisible region angle <(θA− (θa / 2))” and “(θA + (θa / 2)) <invisible region. Angle <= 360 ”. When the visible region extends across the true north line, the invisible region angle is “(θA + (θa / 2)) <invisible region angle <(θA− (θa / 2))”.

  Then, the shooting object determination unit 113 sorts the object information in the object information storage unit 111 in the order of objects closer to the terminal device 13 using the acquired position information (X, Y). The sort process is performed by the process described with reference to FIG. As a result of the sorting process, the shooting object determining unit 113 acquires an object temporary management table as shown in FIG. Each record of the object temporary management table has an object identifier and a distance. The distance is the distance between the point indicated by the position information (X, Y) and the point indicated by the object position information.

  Next, the shooting object determination unit 113 determines whether or not the object is a shooting object from a higher-order record in the object temporary management table.

That is, the shooting object determination unit 113 reads the object identifier “1” of the first record in the temporary object management table. Then, the object information management table is searched using the object identifier “1” as a key, and region information “(x ₁ , y ₁ ) (x ₂ , y ₂ ) (x ₃ , y ₃ ) (x ₄ , y ₄ ) ”.

Next, the shooting object determination unit 113 receives the received terminal information (X, Y, θA, θa) and the acquired area information “(x ₁ , y ₁ ) (x ₂ , y ₂ ) (x ₃ , y ₃ ) (x ₄ , y ₄ ) ”to determine whether or not the first object is a shooting object.

First, the shooting object determination unit 113 acquires the _first point (x ₁ , y ₁ ) in the area information of the object to be determined (object with the object identifier “1”).

Next, the shooting object determination unit 113 calculates an angle (θ1) between the _first point (x ₁ , y ₁ ) and the position information (X, Y), and temporarily stores the angle (θ1) in the buffer. . Similarly, the shooting object determination unit 113 sets each point from the second point to the position information (X, Y) in the region information of the object to be determined (object having the object identifier “1”). The angles (θ2, θ3, θ4) are calculated, and the angles (θ2, θ3, θ4) are also temporarily stored in the buffer. (Θ1, θ2, θ3, θ4) is stored in the buffer.

  Next, the shooting object determination unit 113 acquires the minimum angle (for example, (θ1)) and the maximum angle (for example, (θ4)) from the angles (θ1, θ2, θ3, θ4) temporarily stored in the buffer. To do.

  Next, the imaging object determination unit 113 determines that all the acquired areas of the minimum angle (θ1) and the maximum angle (θ4) are already registered angle areas (“(θA− (θa / 2)) <= visible. It is determined whether or not the region ≦ = (θA + (θa / 2)) ”. Here, it is assumed that the shooting object determination unit 113 determines that the acquired area of the minimum angle (θ1) and the maximum angle (θ4) is a visible area. Then, the shooting object determination unit 113 determines that the object with the object identifier “1” is a shooting object.

  Next, the shooting object determining unit 113 sets “θ1 <= invisible region <= θ4” as a new invisible region. That is, the imaging object determination unit 113, for example, “0 <= invisible area angle <(θA− (θa / 2))” and “(θA + (θa / 2)) <invisible area angle <= 360” and “θ1”. The invisible area angle obtained by merging <= invisible area <= θ4 ”is registered in the recording medium. Here, “merging” means adding “θ1 <= invisible region <= θ4” to the invisible region so far.

Next, the shooting object determination unit 113 stores the object identifier “1” of the first object in a storage medium (not shown).
Next, the shooting object information determination unit 113 determines that a visible area exists from the registered invisible area or visible area.

Next, the shooting object determination unit 113 acquires the second object identifier “17” of the temporary object management table, searches the object information management table using the identifier “17” as a key, and obtains the object identifier “17”. The paired region information (for example, “(x ₁₀₁ , y ₁₀₁ ) (x ₁₀₂ , y ₁₀₂ ) (x ₁₀₃ , y ₁₀₃ ) (x ₁₀₃ , y ₁₀₃ )”) is acquired.

Next, the shooting object determination unit 113 receives the received terminal information (X, Y, θA, θa) and the acquired area information “(x ₁₀₁ , y ₁₀₁ ) (x ₁₀₂ , y ₁₀₂ ) (x ₁₀₃ , y). ₁₀₃ ) (x ₁₀₃ , y ₁₀₃ ) ”is used to determine whether or not the second object is a shooting object.

First, the shooting object determination unit 113 acquires the first point (x ₁₀₁ , y ₁₀₁ ) in the area information of the object to be determined (object with the object identifier “17”).

Next, the shooting object determination unit 113 calculates an angle (θ101) between the first point (x ₁₀₁ , y ₁₀₁ ) and the position information (X, Y), and temporarily stores the angle (θ101) in the buffer. . Similarly, the shooting object determining unit 113 sets each of the fourth point from the second point in the area information of the object to be determined (object having the object identifier “17”) and the position information (X, Y). The angles (θ102, θ103, θ104) are calculated, and the angles (θ102, θ103, θ104) are also temporarily stored in the buffer. (Θ101, θ102, θ103, θ104) is stored in the buffer.

  Next, the shooting object determination unit 113 acquires the minimum angle (for example, (θ101)) and the maximum angle (for example, (θ103)) from the angles (θ101, θ102, θ103, θ104) temporarily stored in the buffer. To do.

Next, the shooting object determination unit 113 determines whether or not all of the acquired areas of the minimum angle (θ101) and the maximum angle (θ103) are already registered angle areas. Then, it is assumed that the shooting object determination unit 113 determines that all the areas of the minimum angle (θ101) and the maximum angle (θ103) are already registered angle areas. Therefore, the shooting object determination unit 113 determines that the object with the object identifier “17” is not a shooting object.
Next, the shooting object information determination unit 113 determines that a visible area exists from the registered invisible area or visible area.

Next, the shooting object determining unit 113 acquires the third object identifier “3” of the temporary object management table, searches the object information management table using the identifier “3” as a key, and sets the object identifier “17” as “17”. The paired region information “(x ₁₁ , y ₁₁ ) (x ₁₂ , y ₁₂ ) (x ₁₃ , y ₁₃ ) (x ₁₄ , y ₁₄ )” is acquired.

Next, the shooting object determination unit 113 receives the received terminal information (X, Y, θA, θa) and the acquired area information “(x ₁₁ , y ₁₁ ) (x ₁₂ , y ₁₂ ) (x ₁₃ , y). ₁₃ ) (x ₁₄ , y ₁₄ ) ”, it is determined whether or not the third object is a shooting object.

First, the shooting object determination unit 113 obtains the first point (x ₁₁ , y ₁₁ ) in the area information of the object to be determined (object with the object identifier “3”).

Next, the shooting object determination unit 113 calculates an angle (θ11) between the first point (x ₁₁ , y ₁₁ ) and the position information (X, Y), and temporarily stores the angle (θ11) in the buffer. . Similarly, the shooting object determination unit 113 sets each point from the second point to the position information (X, Y) in the region information of the object to be determined (object with the object identifier “3”). The angles (θ12, θ13, θ14) are calculated, and the angles (θ12, θ13, θ14) are also temporarily stored in the buffer. (Θ11, θ12, θ13, θ14) is stored in the buffer.

  Next, the shooting object determination unit 113 acquires the minimum angle (for example, (θ11)) and the maximum angle (for example, (θ14)) from the angles (θ11, θ12, θ13, θ14) temporarily stored in the buffer. To do.

  Next, the shooting object determination unit 113 determines whether or not all of the acquired areas of the minimum angle (θ11) and the maximum angle (θ14) are already registered angle areas. Here, the shooting object determination unit 113 compares the acquired area of the minimum angle (θ11) and the maximum angle (θ14) with the angle of the already registered invisible area. Then, it is assumed that the shooting object determination unit 113 determines that the acquired area of the minimum angle (θ11) and the maximum angle (θ14) is a visible area. Then, the shooting object determination unit 113 determines that the object with the object identifier “3” is a shooting object.

  Next, the shooting object determination unit 113 sets “θ11 <= invisible region <= θ14” as a new invisible region.

Next, the shooting object determination unit 113 stores the object identifier “3” of the third object in a storage medium (not shown).
Next, the shooting object information determination unit 113 determines that a visible area exists from the registered invisible area or visible area.

  The above processing is repeated for the fourth and subsequent objects. Then, after determining that the object identified by the object identifier “25” is a shooting object, the shooting object information determination unit 113 determines that there is no visible area from the registered invisible area or visible area. Suppose that Finally, it is assumed that a set “1, 3, 25” of identifiers of shooting objects is obtained.

Next, the shooting object information with the object identifier “1” is configured. That is, the shooting object information acquisition unit 115 obtains the object name “coffee shop A” of the object identifier “1”, the URL “http://www.coffee-A.co.jp”, the position information (from the object information management table). xa, ya) and area information “(x ₁ , y ₁ ) (x ₂ , y ₂ ) (x ₃ , y ₃ ) (x ₄ , y ₄ )” are read out. Next, the shooting object information acquisition unit 115 reads the distance “100 m” from the object temporary management table. Then, the shooting object information acquisition unit 115 has the object name “coffee shop A” of the object identifier “1”, the URL “http://www.coffee-A.co.jp”, the position information (xa, ya), the area. The photographing object information having the information “(x ₁ , y ₁ ) (x ₂ , y ₂ ) (x ₃ , y ₃ ) (x ₄ , y ₄ )” and the distance “100 m” is configured and temporarily stored in the buffer. .

  Similarly, the shooting object information acquisition unit 115 configures shooting object information with the object identifier “3” and the object identifier “25” and adds the shooting object information to the buffer. Then, the shooting object information acquisition unit 115 obtains a shooting object information management table shown in FIG.

  Next, the server transmission unit 116 transmits one or more pieces of shooting object information (shooting object information management table shown in FIG. 19) temporarily stored in the buffer to the terminal device 13.

  Next, the receiving unit 1306 receives one or more shooting object information in response to the transmission of the terminal information.

  Next, the output unit 1309 reads “current layer = object identification layer” from the current layer storage unit 1301.

Next, the output information acquisition unit 1308 includes shooting object information (object identifier “1”, object name “coffee shop A”, URL “http: // www. coffee-A.co.jp ", the position information (xa, ya), the area information" _{(x 1, y 1) (} x 2, y 2) (x 3, y 3) (x 4, y 4) ", Distance "100 m").

Then, the area output unit 13092 of the output unit 1309 uses the area information “(x ₁ , y ₁ ) (x ₂ , y ₂ ) (x ₃ , y ₃ ) (x ₄ , y ₄ )” of the shooting object, Determine the frame size.

  Next, the area output unit 13092 configures a frame that matches the determined frame size. Then, the region output unit 13092 outputs the configured frame to the region indicated by the region information.

  The distance output unit 13093 outputs the distance “100 m” included in the shooting object information in association with the frame. An example of such output is shown in FIG. Here, the frame is a rectangle. Further, the height of the frame matches the height of the display. That is, the height of the frame of all objects is constant.

  In the above description, only the information of the shooting object that is the closest to the terminal device 13 is output. However, only the information of the shooting object existing at the center of the terminal device 13 may be output. In this case, the shooting object information includes shooting object direction information (the middle angle (average) between the maximum angle and the minimum angle). Then, the shooting object having the smallest difference between the direction information of the shooting object and the direction information included in the terminal information is selected, and the information of the shooting object is output. Furthermore, it goes without saying that the area output unit 13092 may output information on all the shooting objects. An output example in such a case is shown in FIG. In FIG. 21, only the distance of the shooting object with the shortest distance (or closest to the center) is output, but it goes without saying that the distances of all the shooting objects may be output.

  It is assumed that the user inputs an instruction to fix the output in the screen output state of FIG. Then, the output fixing unit 1311 fixes the video and area (screen in FIG. 20) being output by the output unit 1309. This screen fixing process is a known technique. Further, the screen fixing process is performed by, for example, a process in which the image of the VRAM is not changed and the operation of the camera unit 1304 is stopped.

  Next, it is assumed that the user inputs (touches) an instruction to the frame of the shooting object (object on which the frame is displayed) having the object identifier “1” in a state where the screen is fixed. Then, the second accepting unit 13022 accepts an input for an area (here, a rectangular frame). Since the screen is fixed, even if the terminal device 13 is moved, the display on the screen does not change and it is easy for the user to operate.

  Next, information related to the object is output by processing as described below. That is, the object identifier acquisition unit 1312 acquires the object identifier for the area corresponding to the input received by the second receiving unit 13022 from the shooting object information. The object identifier here is assumed to be “URL”. That is, the object identifier acquisition unit 1312 acquires the URL “http://www.coffee-A.co.jp”.

  Note that when an instruction is input to the frame of the shooting object (object on which the frame is displayed), at least temporarily, the current layer of the current layer storage unit 1301 may be changed to the web search layer. good.

  Next, the object identifier transmission unit 1313 changes the URL “http://www.coffee-A.co.jp” to the URL “http://www.coffee-A.co.jp” using the acquired URL “http://www.coffee-A.co.jp”. The corresponding web server device 12 is accessed.

  Then, the object related information acquisition unit 122 of the web server device 12 acquires object related information corresponding to the URL “http://www.coffee-A.co.jp”. Here, the acquired object related information is the object related information shown in FIG.

  Next, the object related information transmission unit 123 of the web server device 12 transmits the object related information illustrated in FIG. 15 to the terminal device 13.

  Next, the object related information receiving unit 1307 of the terminal device 13 receives the object related information shown in FIG.

  Since the received object related information includes symbol information (symbol image), the output unit 1309 reduces the symbol image to the size of the frame size. Then, the object related information output unit 13094 arranges the reduced symbol image in the frame area. FIG. 22 shows an example of such output. The reduced symbol image is 2201.

  The output unit 1309 displays the video acquired by the camera unit 1304 on the first screen (first display unit 13141), and the object related information output unit 13094 receives the received object related information (here, symbol It is preferable to display the image) on the second display means 13142. An example of such output is shown in FIG. In FIG. 23, the object related information output unit 13094 displays a frame and a distance on the first display unit 13141 and displays a symbol image on the second display unit 13142.

  Furthermore, it is assumed that the terminal device 13 is in the display state of FIG. Next, it is assumed that the user inputs an instruction to set the current layer as the web search layer. Then, the third receiving unit 13023 determines that the received input is an input for setting the current layer. Next, the current layer changing unit 1310 changes the current layer in the current layer storage unit 1301 to the web search layer according to the received input.

  Next, the user inputs an object acquisition instruction to obtain information on the object in the video from the state of the terminal device 13 in FIG.

  Next, the terminal information acquisition unit 1303 configures terminal information (X, Y, θA, θa) by the processing described above.

  Next, the transmission unit 1305 transmits the configured terminal information (X, Y, θA, θa) to the server device 11.

  Then, the server reception unit 112 receives terminal information (X, Y, θA, θa) from the terminal device 13.

  Next, the server reception unit 112 determines the shooting object (ID [1, 3, 25]) by the processing described above. Then, the shooting object information acquisition unit 115 obtains a shooting object information management table shown in FIG.

  Next, the server transmission unit 116 transmits one or more pieces of shooting object information (shooting object information management table shown in FIG. 19) temporarily stored in the buffer to the terminal device 13.

  Next, the receiving unit 1306 receives one or more shooting object information in response to the transmission of the terminal information.

  Next, the output unit 1309 reads “current layer = web search layer” from the current layer storage unit 1301.

Next, the output information acquisition unit 1308 includes shooting object information (object identifier “1”, object name “coffee shop A”, URL “http: // www. coffee-A.co.jp ", the position information (xa, ya), the area information" _{(x 1, y 1) (} x 2, y 2) (x 3, y 3) (x 4, y 4) ", Distance "100 m").

Then, the area output unit 13092 of the output unit 1309 uses the area information “(x ₁ , y ₁ ) (x ₂ , y ₂ ) (x ₃ , y ₃ ) (x ₄ , y ₄ )” of the shooting object, Determine the frame size.

  Next, the area output unit 13092 configures a frame that matches the determined frame size. Then, the region output unit 13092 outputs the configured frame to the region indicated by the region information.

  The distance output unit 13093 outputs the distance “100 m” included in the shooting object information in association with the frame.

  Next, the object related information output unit 13094 displays the received object related information (here, a symbol image) on the second display unit 13142. An example of such output is shown in FIG. The object-related information output unit 13094 may output as shown in FIG. 22 instead of FIG. That is, the output mode of the object related information is not limited.

  Further, since the object area is a two-dimensional area, in FIGS. 20, 21, 22, and 23, the frame is a full-sized frame. However, for example, the size of the object region (frame size) may be calculated by a decreasing function using the distance as a parameter, and the frame may be output. That is, the larger the distance, the smaller the area of the object corresponding to the distance may be output. In such a case, for example, the output is a frame as shown in FIG. In FIG. 24, the height of the frame is inversely proportional to the distance.

  (Specific example 2)

  Now, the object information storage unit 111 of the server apparatus 11 holds the object information management table shown in FIG. The object information management table is a table that manages object information on the map. The object information management table manages one or more records having attribute values such as “object identifier”, “object name”, “URL”, “location information”, and “region information”. In FIG. 25, the difference from the object information management table of FIG. 14 is that each point constituting the area information has height information (z).

  Moreover, the object relevant information storage part 121 of the web server apparatus 12 has the object relevant information shown in FIG. 15, for example.

  Further, it is assumed that the terminal device 13 is a terminal having two displays. The terminal device 13 may be a terminal having one display or a terminal having three or more displays.

  Furthermore, it is assumed that the current layer storage unit 1301 of the terminal device 13 stores “current layer = camera layer”.

  In this case, it is assumed that the user has activated the terminal device 13. Then, the accepting unit 1302 accepts an activation instruction. Then, the camera unit 1304 reads “current layer = camera layer” from the current layer storage unit 1301. Then, the camera unit 1304 activates the camera. Then, as shown in FIG. 16, an image of the outside world is output to the terminal device 13.

  Next, it is assumed that the user has input an instruction to make the current layer an object identification layer. Then, the third receiving unit 13023 determines that the received input is an input for setting the current layer. Next, the current layer changing unit 1310 changes the current layer of the current layer storage unit 1301 to the object identification layer according to the received input.

  Next, the user inputs an object acquisition instruction to obtain information on the object in the video from the state of the terminal device 13 in FIG. The object acquisition instruction is preferably input by touching, but may be input by selection from a menu, input by an input means such as a keyboard, or the like. Next, the first receiving unit 13021 receives an object acquisition instruction.

  Next, the position information acquisition unit 13031 (here, for example, a GPS receiver) of the terminal information acquisition unit 1303 acquires the position information (X, Y, Z) of the terminal device 13. Z is the height.

  Next, the direction information acquisition unit 13032 acquires direction information (θA, θB) indicating the direction in which the terminal device 13 is facing. θA is an angle on a plane, and θB is an angle in the height direction.

  In addition, the terminal information acquisition unit 1303 reads information on the viewing angle of the camera unit 1304 (simply referred to as “viewing angle”). Here, it is assumed that the viewing angles are (θa, θb). The camera unit 1304 stores the viewing angles (θa, θb) in advance. Here, θa is an angle on the plane, and θb is an angle in the height direction. The viewing angles (θa, θb) are shown in FIG.

  Next, the terminal information acquisition unit 1303 acquires terminal information (X, Y) having the acquired position information (X, Y, Z), the acquired direction information (θA, θB), and the viewing angle (θa, θb). , Z, θA, θB, θa, θb).

  Next, the transmission unit 1305 transmits the configured terminal information (X, Y, Z, θA, θB, θa, θb) to the server device 11.

  Then, the server reception unit 112 receives terminal information (X, Y, Z, θA, θB, θa, θb) from the terminal device 13.

  Next, the shooting object determination unit 113 determines a shooting object as follows. That is, the shooting object determination unit 113 acquires position information (X, Y, Z) included in the received terminal information.

  Next, the shooting object determination unit 113 calculates a three-dimensional angle that becomes an invisible region by using the direction information (θA, θB) and the viewing angle (θa, θb) included in the terminal information. The shooting object determination unit 113 acquires a two-dimensional angle “(θA− (θa / 2)) <= visible region <= (θA + (θa / 2))” that becomes a visible region. Then, the angle “(θB− (θb / 2)) <= visible region <= (θB + (θb / 2)))” which becomes the visible region is acquired. Then, the area that is not the visible area is accumulated as an invisible area.

Then, the shooting object determination unit 113 sorts the object information in the object information storage unit 111 in the order of objects closer to the terminal device 13 using the acquired position information (X, Y, Z). The sort process is performed by the process described with reference to FIG. Here, using (X, Y) of the position information (X, Y, Z) and the position information (for example, (x _a , y _a )) of each object, the terminal device 13 and each Calculate the distance to the object and sort using the distance as a key. As a result of the sorting process, the shooting object determination unit 113 acquires an object temporary management table as shown in FIG. Each record of the object temporary management table has an object identifier and a distance. The distance is a distance between a point indicated by the position information (X, Y, Z) (however, Z is not used) and a point indicated by the position information of the object.

  Next, the shooting object determination unit 113 determines whether or not the object is a shooting object from a higher-order record in the object temporary management table.

That is, the shooting object determination unit 113 reads the object identifier “1” of the first record in the temporary object management table. Then, the object information management table is searched using the object identifier “1” as a key, and region information “(x ₁ , y ₁ , z ₁ ) (x ₂ , y ₂ , z) paired with the object identifier“ 1 ”is searched. ₂ ) (x ₃ , y ₃ , z ₃ ) (x ₄ , y ₄ , z ₄ ) ”.

Next, the shooting object determination unit 113 receives the received terminal information (X, Y, Z, θA, θB, θa, θb) and the acquired area information “(x ₁ , y ₁ , z ₁ ) (x ₂ ). , Y ₂ , z ₂ ) (x ₃ , y ₃ , z ₃ ) (x ₄ , y ₄ , z ₄ ) ”is used to determine whether or not the first object is a shooting object.

First, the shooting object determination unit 113 acquires the _first point (x ₁ , y ₁ , z ₁ ) in the area information of the object to be determined (object with the object identifier “1”).

Next, the shooting object determination unit 113 calculates an angle (θ11) on the plane between the _first point (x ₁ , y ₁ , z ₁ ) and the position information (X, Y, Z), and the angle ( θ11) is temporarily stored in the buffer. In addition, the shooting object determination unit 113 calculates an angle (θ12) in the height direction between the _first point (x ₁ , y ₁ , z ₁ ) and the position information (X, Y, Z), and calculates the angle ( θ12) is temporarily stored in the buffer. Similarly, the shooting object determination unit 113 and the fourth point from the second point in the area information of the object to be determined (object with the object identifier “1”) and position information (X, Y, Z) Are calculated on the plane (θ12, θ13, θ14), and the angles (θ12, θ13, θ14) are also temporarily stored in the buffer. (Θ11, θ12, θ13, θ14) is stored in the buffer. Further, similarly, the shooting object determination unit 113 sets each of the fourth point from the second point in the area information of the object to be determined (object having the object identifier “1”) and the position information (X, Y, The angle (θ22, θ23, θ24) in the height direction with respect to Z) is calculated, and the angles (θ22, θ23, θ24) are also temporarily stored in the buffer. (Θ21, θ22, θ23, θ24) is stored in the buffer.

  Next, the shooting object determining unit 113 determines the minimum angle (for example, (θ11)) and the maximum angle (for example, (θ14)) on the plane from the angles (θ11, θ12, θ13, θ14) temporarily stored in the buffer. And get.

  Further, the shooting object determination unit 113 acquires the maximum angle (θ24) in the height direction from the angles (θ21, θ22, θ23, θ24) temporarily stored in the buffer.

  Next, the shooting object determination unit 113 determines whether or not all of the acquired areas of the minimum angle (θ11) and the maximum angle (θ14) on the plane are already registered areas. Here, it is assumed that the shooting object determination unit 113 determines that the acquired area of the minimum angle (θ11) and the maximum angle (θ14) is a visible area. Then, the shooting object determination unit 113 determines that the object with the object identifier “1” is a shooting object.

  Here, for example, the shooting object determining unit 113 may determine that a part of the acquired area of the minimum angle (θ11) and the maximum angle (θ14) on the plane is not an already registered area. It is determined that it is in the visible region. That is, the shooting object determination unit 113 determines that the object with the object identifier “1” is a shooting object. Further, for example, even when the shooting object determination unit 113 determines that all of the acquired areas of the minimum angle (θ11) and the maximum angle (θ14) on the plane are already registered angle areas, When the maximum angle (θ24) in the vertical direction is larger than the registered maximum angle in the height direction, it is determined that the region is the visible region in the region of the minimum angle (θ11) and the maximum angle (θ14) on the plane. That is, the shooting object determination unit 113 determines that the object with the object identifier “1” is a shooting object.

  Next, the shooting object determination unit 113 sets “θ11 <= invisible region <= θ14” as a new invisible region. That is, the imaging object determination unit 113, for example, “0 <= invisible area angle <(θA− (θa / 2))” and “(θA + (θa / 2)) <invisible area angle <= 360” and “θ11”. The invisible area angle on the plane obtained by merging <= invisible area <= θ14 ”is registered in the recording medium. If the registered maximum angle in the height direction is smaller than (θ24), the shooting object determination unit 113 registers the maximum angle (θ24) in the height direction in association with the area on the plane.

Next, the shooting object determination unit 113 stores the object identifier “1” of the first object in a storage medium (not shown).
Next, it is assumed that the shooting object information determination unit 113 determines that there is a visible area from the registered invisible area or visible area.

The above processing is also performed on the second and subsequent objects in FIG. 18 to determine the shooting object.
Then, the shooting object information determination unit 113 repeats the above inspection (inspection for determining a shooting object) until it is determined from the registered invisible area or visible area that there is no visible area. Then, it is assumed that the shooting object information acquisition unit 115 has acquired object identifiers “1”, “3”, and “25” of the shooting object. Then, it is assumed that the shooting object information acquisition unit 115 has obtained the shooting object information management table shown in FIG.

  Next, the server transmission unit 116 transmits one or more pieces of shooting object information (shooting object information management table shown in FIG. 19) temporarily stored in the buffer to the terminal device 13.

  Next, the receiving unit 1306 receives shooting object information in response to transmission of terminal information.

  Next, the output unit 1309 reads “current layer = object identification layer” from the current layer storage unit 1301.

Next, the output information acquisition unit 1308 includes shooting object information (object identifier “1”, object name “coffee shop A”, URL “http: // www. coffee-A.co.jp ", the position information (xa, ya), the area information" _{(x 1, y 1, z} 1) (x 2, y 2, z 2) (x 3, y 3, z 3) (X ₄ , y ₄ , z ₄ ) ”, distance“ 100 m ”).

Then, the area output unit 13092 of the output unit 1309 displays the area information “(x ₁ , y ₁ , z ₁ ) (x ₂ , y ₂ , z ₂ ) (x ₃ , y ₃ , z ₃ ) ( x ₄ , y ₄ , z ₄ ) ”determines the frame size including the height.

  Next, the area output unit 13092 configures a frame that matches the determined frame size. Then, the region output unit 13092 outputs the configured frame to the region indicated by the region information.

  The distance output unit 13093 outputs the distance “100 m” included in the shooting object information in association with the frame. An example of such output is shown in FIG. Here, the frame is a rectangle.

  In the above description, only the information of the shooting object that is the closest to the terminal device 13 is output. However, only the information of the shooting object existing at the center of the terminal device 13 may be output. In this case, the shooting object information includes shooting object direction information (the middle angle (average) between the maximum angle and the minimum angle). Then, the shooting object having the smallest difference between the direction information of the shooting object and the direction information included in the terminal information is selected, and the information of the shooting object is output. Furthermore, it goes without saying that the area output unit 13092 may output information on all the shooting objects. An example of output in such a case is shown in FIG. In FIG. 28, only the distance of the shooting object having the shortest distance (or closest to the center) is output, but it is needless to say that the distances of all the shooting objects may be output.

  The subsequent processing may be the same as the processing described in the first specific example.

As described above, according to the present embodiment, it is possible to recognize an object photographed by a camera and output information related to the object. Therefore, the user can very easily obtain information such as buildings, landmarks, and parks viewed with his / her own eyes.
Further, according to the present embodiment, high-speed processing can be performed by appropriately terminating the processing for detecting a shooting object. Therefore, the terminal device can be used for a navigation system.

  Further, according to the present embodiment, since the height of the object is also taken into account, for example, information on a building that is visible behind a certain building can be obtained.

  Further, according to the present embodiment, the output object information is reduced or enlarged in accordance with the size of the display area (frame) on the camera, so that the size of the object can be seen at a glance.

  In addition, according to the present embodiment, the distance from the camera to the object is calculated and output, so that the distance to the object being photographed by the camera can be easily grasped and whether or not to go there It becomes support of the action.

  Further, according to the present embodiment, even when object height information is not managed, it is possible to recognize an object being photographed by a camera and output information related to the object.

  Further, according to the present embodiment, appropriate information on an object can be obtained by acquiring and outputting object-related information from a web server device.

  Further, according to the present embodiment, the object related information is output on one of the two screens, so that the user can view the object related information without disturbing the video of the camera.

  In addition, according to the present embodiment, when one object is selected from a plurality of objects and information on the selected object is output on the screen, a large number of objects exist on the screen. However, it is possible to obtain information about the object that the user wants.

  Furthermore, according to the present embodiment, by fixing the video on the screen, the user can operate the terminal device 13 with an easy posture when it is desired to obtain object-related information.

  Note that the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. Needless to say, this software or a recording medium storing this software may be distributed as a computer program product. This also applies to other embodiments in this specification. In addition, the software which implement | achieves the server apparatus 11 in this Embodiment is the following programs. That is, this program stores at least one piece of object information having an object identifier for identifying an object and area information indicating an area in which the object exists in a storage medium. A server reception unit that receives terminal information including position information indicating the position of the terminal device and direction information indicating a direction in which the terminal device is facing, position information included in the terminal information, and a direction included in the terminal information A shooting object that applies information and a viewing angle stored in advance or possessed by the terminal information to region information of each object information of the storage medium to determine one or more objects being shot by the terminal device A determination unit and one or more objects corresponding to the one or more objects determined by the shooting object determination unit; To obtain one or more shooting object information including area information from the storage medium, and to function as a server transmission unit that transmits the shooting object information to the terminal device. Program.

  In the above program, the area information includes two-dimensional area information indicating a two-dimensional area on a map and height information indicating the height of an object. Using the position information and the direction information having and the viewing angle stored in advance or possessed by the terminal information, a shooting area in a three-dimensional space is acquired, and the shooting area is set as each object information of the storage medium. It is preferable to apply to the area information that the terminal device has and determine one or more objects that the terminal device is photographing.

  In the above program, the shooting object determination unit determines the distance between each object determined by the shooting object determination unit and the terminal device from the object information of each object and the position information of the terminal information. A distance calculation unit that calculates for each object is further included, and the shooting object information acquisition unit includes area information among one or more object information corresponding to each of the one or more objects determined by the shooting object determination unit. Is acquired from the storage medium, and it is preferable to configure one or more shooting object information including information including the region information and the distance calculated by the distance calculation unit.

  Further, in the above program, the shooting object determination unit is the shooting object of the object that is present at the closest position in the video shot by the camera unit, or the most of the video shot by the camera unit. It is preferable to determine the shooting object of the object located closer to the center or the shooting object of the object having the largest area in the video shot by the camera unit.

  In the program, it is preferable that the server transmission unit transmits the shooting object information to the terminal device or one or more of the web server devices. When the server transmission unit transmits the shooting object information to one or more web server devices, the object related information is transmitted from the web server device (directly or indirectly) to the terminal device.

  The software that implements the terminal device 13 in the present embodiment is the following program. In other words, this program uses a computer to acquire terminal information including position information indicating the position of the terminal device and direction information indicating the direction in which the terminal device is facing, and a camera that acquires video. An output unit including a camera video output unit that outputs video acquired by the camera unit, a transmission unit that transmits the terminal information acquired by the terminal information acquisition unit to the server device, and transmission of the terminal information Corresponding to the receiving unit that receives the one or more shooting object information, and an output information acquisition unit that acquires output information including one or more region information included in the one or more shooting object information received by the receiving unit; And the output unit outputs region information included in the one or more pieces of output information acquired by the output information acquisition unit, superimposed on the video output by the output unit. Program for functioning as comprising a force means, a.

  In the above program, the area output means can visually distinguish an area having a size corresponding to the area information included in the one or more pieces of output information acquired by the output information acquisition unit from other areas. In addition, it is preferable to output the video output from the output unit so as to overlap it.

  In the program, the output information acquisition unit acquires output information including one or more area information and one or more distances included in the shooting object information received by the reception unit, and the output unit includes the output It is preferable to further include distance output means for outputting a distance included in the one or more pieces of output information acquired by the information acquisition unit.

  In the above program, it is preferable that the area output unit outputs the object area corresponding to the distance smaller as the distance included in the one or more pieces of output information acquired by the output information acquisition unit is larger. It is.

  The program further includes an object related information receiving unit that receives object related information, and the output unit further includes an object related information output unit that outputs the object related information received by the object related information receiving unit. It is preferable to do.

  Further, the program includes a first display unit and a second display unit which are two separate display units, and the camera video output unit sends the video acquired by the camera unit to the first display unit. It is preferable that the object related information output means displays the object related information received by the object related information receiving unit on the second display means.

  In the above program, the output unit outputs the area information so that only the area information of one object can be visually distinguished from the other area information without outputting the area information of the other object. It is preferable to output the image superimposed on the current image.

  In the above program, it is preferable that the object related information output means outputs only the object related information of one object without outputting the object related information of another object.

  In the above program, further comprising: a first receiving unit that receives a user instruction; and an output fixing unit that fixes an image and a region being output by the output unit when the receiving unit receives the instruction. Is preferred.

  In the above program, the shooting object information includes an object identifier, and obtains an object identifier for an area corresponding to the input received by the second receiving means and a second receiving means for receiving an input to the fixed area. An object identifier acquisition unit that transmits the object identifier acquired by the object identifier acquisition unit to any one of the one or more web server devices, the object related information reception unit, Corresponding to the transmission of the object identifier, it is preferable that the object-related information is received, and the transmission of the object identifier is performed by an instruction from the outside.

  In addition, a camera layer that is a layer that outputs the video acquired by the camera unit, an object identification layer that is a layer that outputs an area corresponding to an object, and a web search layer that is a layer that outputs object-related information are stored in a storage medium. Among the plurality of layers including, the current layer that is the current layer is stored, and in the above program, according to the input received by the third receiving unit, the third receiving unit that receives the input to set the current layer, A current layer changing unit for changing a current layer of the storage medium, and when the current layer is a camera layer, the camera video output means outputs the video acquired by the camera unit, and the current layer In the case of an object identification layer, the camera video output means The video output by the camera unit is output, and the region output means outputs the image superimposed on the video output by the output unit so that the region can be visually distinguished from other regions, When the current layer is a web search layer, the camera video output means outputs the video acquired by the camera unit, and the area output means can visually distinguish the area from other areas. It is preferable that the video output by the output unit is superimposed and output, and the object related information output means outputs the object related information.

  The software that implements the terminal device 13 may be the following program. That is, this program acquires a video from a terminal information acquisition unit that acquires terminal information including position information indicating the position of the terminal device and direction information indicating a direction in which the terminal device is facing. A camera unit; an output unit including a camera video output unit that outputs a video acquired by the camera unit; a transmission unit that transmits terminal information acquired by the terminal information acquisition unit to the server device; and In response to transmission, a program for causing a reception unit to receive the shooting object information and an output unit to output the shooting object information received by the reception unit to function.

(Embodiment 2)
In the present embodiment, a case where the terminal device operates stand-alone will be described. It is assumed that the terminal device accesses the web server device as necessary.

  FIG. 29 is a conceptual diagram of the information system 2. The information system 2 includes a web server device 12 and a terminal device 23. The terminal device 23 may be a terminal having a single-screen display or a terminal having a multi-screen display. The terminal device 23 has a camera function and can output an area of an object (such as a building) in the captured video, information related to the object, or the like when shooting an external landscape or a building. Moreover, the web server apparatus 12 and the terminal device 23 are normally communicable.

  FIG. 30 is a block diagram of the information system 2 in the present embodiment. The terminal device 23 includes an object information storage unit 111, a shooting object determination unit 113, a distance calculation unit 114, a shooting object information acquisition unit 115, a current layer storage unit 1301, a reception unit 1302, a terminal information acquisition unit 1303, a camera unit 1304, an object A related information receiving unit 1307, an output information acquiring unit 1308, an output unit 1309, a current layer changing unit 1310, an output fixing unit 1311, an object identifier acquiring unit 1312, an object identifier transmitting unit 1313, and a display unit 1314 are provided.

  Next, operation | movement of the terminal device 23 which comprises the information system 2 is demonstrated using the flowchart of FIG. In the flowchart of FIG. 31, only steps that do not appear in the flowchart described above will be described.

  (Step S3101) The shooting object information acquisition unit 115 acquires information including area information of the i-th shooting object from the object information storage unit 111. Then, the shooting object information acquisition unit 115 acquires the calculated distance of the i-th shooting object. Next, the shooting object information acquisition unit 115 configures the i-th shooting object information acquired from the object information storage unit 111 and shooting object information having a distance.

  Note that in the flowchart of FIG. 31, the processing ends due to power-off or a processing end interrupt.

  Hereinafter, the specific operation of the information system 2 in the present embodiment is the same as the operation of the information system 1. Needless to say, in the information system 2, the terminal device and the server device do not communicate with each other, and the terminal device 23 is stand-alone and recognizes the object being photographed by the camera. And the terminal device 23 can acquire the information relevant to the object currently image | photographed with the camera from the web server apparatus 12, and can output it.

  That is, the terminal device 23 indicates an object information storage unit that can store one or more object information having an object identifier for identifying an object and region information indicating a region where the object exists, and indicates the position of the terminal device. A terminal information acquisition unit that acquires terminal information including position information and direction information indicating a direction in which the terminal device is facing; a position information included in terminal information acquired by the terminal information acquisition unit; and the terminal information A shooting object determination unit that applies the direction information and the viewing angle stored in advance to the area information of each object information in the object information storage unit and determines one or more objects being shot by the terminal device And one or more pieces of object information corresponding to one or more objects decided by the shooting object decision unit Among them, one or more shooting object information including area information is acquired from the object information storage unit, a shooting object information acquisition unit, a camera unit that acquires a video, and a camera video output that outputs a video acquired by the camera unit And an output unit including an area output means for outputting area information included in the one or more shooting object information so as to be superimposed on the video. In addition, the shooting object information determination unit inspects the objects in order from the shortest distance from the terminal device 23, and sets information on an area that becomes invisible by shooting the object previously determined that the terminal apparatus 23 is shooting. It is preferable to end the inspection as to whether or not the object has been photographed by the terminal device when the visible region runs out.

  In addition, the terminal device 23 indicates an object information storage unit that can store one or more object information including an object identifier for identifying an object and region information indicating a region in which the object exists, and indicates the position of the terminal device. A terminal information acquisition unit that acquires terminal information including position information and direction information indicating a direction in which the terminal device is facing; a position information included in terminal information acquired by the terminal information acquisition unit; and the terminal information A shooting object determination unit that applies the direction information and the viewing angle stored in advance to the area information of each object information in the object information storage unit and determines one or more objects being shot by the terminal device And one or more object information corresponding to each one or more objects determined by the shooting object determination unit In other words, one or more shooting object information including an object identifier is acquired from the object information storage unit, a shooting object information acquisition unit, a camera unit that acquires video, and a video acquired by the camera unit is output. A terminal device including an output unit that outputs the above-described shooting object information may be used. In addition, the shooting object information determination unit inspects the objects in order from the shortest distance from the terminal device 23, and sets information on an area that becomes invisible by shooting the object previously determined that the terminal apparatus 23 is shooting. It is preferable to end the inspection as to whether or not the object has been photographed by the terminal device when the visible region runs out.

As described above, according to the present embodiment, it is possible to recognize an object photographed by a camera and output information related to the object.
Further, according to the present embodiment, high-speed processing can be performed by appropriately terminating the processing for detecting a shooting object. Therefore, the terminal device can be used for a navigation system.

  Further, according to the present embodiment, since communication with the server device is not necessary, it is possible to display appropriate information for the object in the video imaged by the camera at high speed.

  Furthermore, the software that implements the terminal device according to the present embodiment is the following program. That is, this program stores one or more object information having an object identifier for identifying an object and area information indicating an area in which the object exists in the storage medium, and the computer stores the position of the terminal device. Terminal information acquisition unit that acquires terminal information including position information indicating the direction of the terminal device, direction information indicating the direction in which the terminal device is facing, position information included in the terminal information acquired by the terminal information acquisition unit, and the terminal A shooting object determination unit that applies direction information included in information and a pre-stored viewing angle to region information of each object information of the storage medium, and determines one or more objects being shot by the terminal device And one or more pieces of object information corresponding to each of the one or more objects determined by the shooting object determination unit A shooting object information acquisition unit that acquires one or more shooting object information including area information from the storage medium; a camera unit that acquires a video; a camera video output unit that outputs a video acquired by the camera unit; A program for causing a computer to function as an output unit including an area output unit that outputs area information included in the one or more shooting object information, overlaid on a video.

  In the above program, the area information includes two-dimensional area information indicating a two-dimensional area on a map and height information indicating the height of an object. Using the position information and the direction information having and the viewing angle stored in advance, a shooting area in a three-dimensional space is acquired, and the shooting area is compared with the area information included in each object information of the storage medium It is preferable to determine one or more objects being photographed by the terminal device.

  In the above program, the area output means can visually distinguish an area having a size corresponding to the area information included in the one or more pieces of output information acquired by the output information acquisition unit from other areas. In addition, it is preferable to output the video output from the output unit so as to overlap it.

  In the above program, the shooting object determination unit determines the distance between each object determined by the shooting object determination unit and the terminal device from the object information of each object and the position information of the terminal information. A distance calculation unit that calculates for each object is further included, and the shooting object information acquisition unit includes area information among one or more object information corresponding to each of the one or more objects determined by the shooting object determination unit. Is acquired from the storage medium, and it is preferable to configure one or more shooting object information including information including the region information and the distance calculated by the distance calculation unit.

  In the above program, it is preferable that the output unit further includes distance output means for outputting a distance included in the one or more pieces of shooting object information.

  In the above program, it is preferable that the region output means outputs the object region corresponding to the distance with a smaller distance as the distance included in the one or more pieces of shooting object information is larger.

  The program further includes an object related information receiving unit that receives object related information from an external device, and the output unit outputs object related information that is output by the object related information receiving unit. It is preferable to further comprise means.

  In the above program, the terminal device includes a first display unit and a second display unit which are two separated display units, and the camera video output unit displays the video acquired by the camera unit, It is preferable that the object related information output means displays the object related information received by the object related information receiving unit on the second display means.

  In the above program, the output unit outputs the area information so that only the area information of one object can be visually distinguished from the other area information without outputting the area information of the other object. It is preferable to output the image superimposed on the current image.

  In the above program, it is preferable that the object related information output means outputs only the object related information of one object without outputting the object related information of another object.

  Further, in the above program, the shooting object determination unit is the shooting object of the object that is present at the closest position in the video shot by the camera unit, or the most of the video shot by the camera unit. It is preferable to determine the shooting object of the object located closer to the center or the shooting object of the object having the largest area in the video shot by the camera unit.

  In the above program, further comprising: a first receiving unit that receives a user instruction; and an output fixing unit that fixes an image and a region being output by the output unit when the receiving unit receives the instruction. Is preferred.

  In the above program, the shooting object information includes an object identifier, and obtains an object identifier for an area corresponding to the input received by the second receiving means and a second receiving means for receiving an input to the fixed area. An object identifier acquisition unit that transmits the object identifier acquired by the object identifier acquisition unit to any one of the one or more web server devices, the object related information reception unit, Corresponding to the transmission of the object identifier, it is preferable that the object-related information is received, and the transmission of the object identifier is performed by an instruction from the outside.

  In the above program, the storage layer is a camera layer that outputs a video acquired by the camera unit, an object identification layer that outputs a region corresponding to an object, and a layer that outputs object-related information. Among the plurality of layers including the web search layer, the current layer that is the current layer is stored, and the third receiving unit that receives an input for setting the current layer, and the input received by the third receiving unit, A current layer changing unit for changing a current layer of the storage medium, and when the current layer is a camera layer, the camera video output means outputs the video acquired by the camera unit, and the current layer In the case of an object identification layer, the camera video output means The video output by the camera unit is output, and the region output means outputs the image superimposed on the video output by the output unit so that the region can be visually distinguished from other regions, When the current layer is a web search layer, the camera video output means outputs the video acquired by the camera unit, and the area output means can visually distinguish the area from other areas. It is preferable that the video output by the output unit is superimposed and output, and the object related information output means outputs the object related information.

  Moreover, the following programs may be sufficient as the software which implement | achieves the terminal device 23 in this Embodiment. That is, this program stores one or more object information having an object identifier for identifying an object and area information indicating an area in which the object exists in the storage medium, and the computer stores the position of the terminal device. Terminal information acquisition unit that acquires terminal information including position information that indicates and direction information that indicates a direction in which the terminal device is facing, position information that the terminal information acquired by the terminal information acquisition unit includes, and the terminal information A shooting object determining unit that applies the direction information included in the image and the viewing angle stored in advance to the area information of each object information of the storage medium, and determines one or more objects being shot by the terminal device; Of the one or more pieces of object information corresponding to the one or more objects determined by the shooting object determination unit, A shooting object information acquisition unit that acquires one or more shooting object information including a project identifier from the storage medium; a camera unit that acquires a video; and a video acquired by the camera unit; and the one or more shooting objects A program for causing a computer to function as an output unit that outputs information.

  In this specification, the terminal device is a device having two screens. However, it goes without saying that the terminal device may be a one-screen device as shown in FIG. In FIG. 32, the terminal device is a UMPC (Ultra-Mobile PC) with a camera. In addition, the terminal device has a pseudo see-through screen (Pseudo See-through Screen (touchable)). Further, the terminal device includes an accelerometer. Further, the terminal device includes a GPS receiver.

  Further, in this specification, the geographic object included in the video captured by the camera is recognized by the method shown in FIG. That is, from the position of the camera (view-point), using the viewing angle (view-angle) of the camera and the direction information (view-direction) of the camera, the entire area is made a visible area (target-field) and an invisible area. (No-distance specified). Then, an object close to view-point was inspected and a shooting object was determined. At that time, the inspection was performed while adding the area already determined to be a shooting object to the invisible area. In FIG. 33, first, an object 3303 is determined as a shooting object. Then, the objects 3304 and 3305 that enter the invisible region by the object 3303 are not shooting objects. Next, the object 3306 is determined as a shooting object. That is, in FIG. 33, objects 3303 and 3306 are determined as shooting objects. In addition, as described above, it is determined whether or not a visible area exists from a registered invisible area or a visible area. When it is determined that a visible area does not exist, a geographic object included in a video is recognized. Processing was canceled.

  Further, in the present specification, the terminal device may be for vehicle use, that is, for example, the terminal device may be a so-called navigation terminal.

  FIG. 34 shows the external appearance of a computer that executes the program described in this specification to realize the information system or the like of the above-described embodiment. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 34 is an overview of the computer system 340, and FIG. 35 is a diagram showing the internal configuration of the computer system 340. As shown in FIG.

  34, the computer system 340 includes a computer 341 including an FD drive 3411 and a CD-ROM drive 3412, a keyboard 342, a mouse 343, and a monitor 344.

  35, in addition to the FD drive 3411 and the CD-ROM drive 3412, the computer 341 stores an MPU 3413, a bus 3414 connected to the CD-ROM drive 3412 and the FD drive 3411, and a program such as a bootup program. ROM 3415 for connecting to CPU 3413, RAM 3416 for temporarily storing application program instructions and providing a temporary storage space, and hard disk 3417 for storing application programs, system programs, and data . Although not shown here, the computer 341 may further include a network card that provides connection to the LAN.

  A program that causes the computer system 340 to execute functions such as the information system of the above-described embodiment is stored in the CD-ROM 3501 or the FD 3502, inserted into the CD-ROM drive 3412 or the FD drive 3411, and further stored in the hard disk 3417. May be forwarded. Alternatively, the program may be transmitted to the computer 341 via a network (not shown) and stored in the hard disk 3417. The program is loaded into the RAM 3416 at the time of execution. The program may be loaded directly from the CD-ROM 3501, the FD 3502, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 341 to execute functions such as the information system of the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 340 operates is well known and will not be described in detail.

  In the above program, in a transmission step for transmitting information, a reception step for receiving information, etc., processing performed by hardware, for example, processing performed by a modem or an interface card in the transmission step (only performed by hardware). Not included) is not included.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  Further, in each of the above embodiments, it goes without saying that two or more communication means (a server reception unit, a server transmission unit, etc.) existing in one device may be physically realized by one medium.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the information system according to the present invention has an effect of recognizing an object photographed by a camera and outputting information related to the object, and is useful as an information system or the like.

Conceptual diagram of information system 1 according to Embodiment 1 Block diagram of the information system Block diagram of the terminal device Flow chart for explaining the operation of the server device Flow chart for explaining the operation of the terminal device A flowchart for explaining the operation of the object sort process A flowchart for explaining the operation of the visibility determination process A flowchart for explaining the operation of the visibility determination process Flow chart for explaining the operation of the terminal device A flowchart for explaining the operation of the shooting object information output process A flowchart for explaining the operation of the object related information output process The figure which shows the concept of the process of the information system 1 Diagram explaining the same layer Figure showing the same object information management table The figure which shows the example of the same object related information The figure which shows the output example of the terminal device in the case of the camera layer Conceptual diagram of judgment of the visible region Figure showing the same object temporary management table The figure which shows the photography object information management table The figure which shows the output example of the terminal device The figure which shows the output example of the terminal device The figure which shows the output example of the terminal device The figure which shows the output example of the terminal device The figure which shows the output example of the terminal device Figure showing the same object information management table Diagram explaining the 3D viewing angle The figure which shows the output example of the terminal device The figure which shows the output example of the terminal device Conceptual diagram of information system 2 in the second embodiment Block diagram of the terminal device Flow chart for explaining the operation of the terminal device Diagram showing the configuration and appearance of the terminal device The figure explaining the determination method of the photography object Overview of the computer system The figure which shows the internal configuration of the computer system

Claims (12)

An information system comprising a server device and a terminal device,
The server device
An object information storage unit capable of storing one or more object information having an object identifier for identifying an object and area information indicating an area in which the object exists;
A server reception unit that receives terminal information including position information indicating the position of the terminal device and direction information indicating a direction in which the terminal device is facing from the terminal device;
Apply the position information that the terminal information has, the direction information that the terminal information has, and the viewing angle that is stored in advance or that the terminal information has to the area information of each object information in the object information storage unit, A shooting object determination unit that determines one or more objects that the terminal device is shooting;
A shooting object information acquisition unit that acquires, from the object information storage unit, one or more shooting object information including region information among one or more object information corresponding to each of the one or more objects determined by the shooting object determination unit; ,
A server transmission unit that transmits the shooting object information to the terminal device;
The terminal device
A terminal information acquisition unit that acquires terminal information having position information indicating the position of the terminal device and direction information indicating a direction in which the terminal device is facing;
A camera unit for acquiring images;
An output unit comprising camera video output means for outputting video acquired by the camera unit;
A transmission unit for transmitting the terminal information acquired by the terminal information acquisition unit to the server device;
In response to the transmission of the terminal information, a receiving unit that receives the one or more shooting object information;
An output information acquisition unit that acquires output information including one or more area information included in the one or more shooting object information received by the reception unit;
The output unit is
A region output unit that outputs region information included in the one or more pieces of output information acquired by the output information acquisition unit, overlaid on the video output by the output unit;
The shooting object information determination unit
The objects are inspected in order from the shortest distance from the terminal device, and information on an area that becomes invisible by photographing the object that has been determined as being photographed by the terminal device is registered as an invisible area, and the visible area disappears. An information system that terminates the inspection of whether or not the object is photographed by the terminal device. The region information includes two-dimensional region information indicating a two-dimensional region on the map and height information indicating the height of the object,
The shooting object determination unit
Using the position information and direction information included in the terminal information and the viewing angle stored in advance or included in the terminal information, a shooting area in a three-dimensional space is acquired, and the shooting area is set as the object information. Applying to the region information of each object information of the storage unit, determining one or more objects photographed by the terminal device; and
The objects are inspected in order from the shortest distance from the terminal device, and information on an area that becomes invisible by photographing the object that has been determined as being photographed by the terminal device is registered as an invisible area, and the visible area disappears. 2. The information system according to claim 1, wherein the inspection of whether or not the object is photographed by the terminal device is terminated. The region output means includes
In the video output by the output unit, the region having a size corresponding to the region information included in the one or more pieces of output information acquired by the output information acquisition unit can be visually distinguished from other regions. The information system according to claim 1, wherein the information system is output in a superimposed manner. The server device
A distance for calculating the distance between each object determined by the shooting object determination unit and the terminal device for each object determined by the shooting object determination unit from the object information of each object and the position information of the terminal information Further comprising a calculation unit;
The shooting object information acquisition unit
Of the one or more pieces of object information corresponding to the one or more objects determined by the shooting object determination unit, information including region information is acquired from the object information storage unit, and the information including the region information and the distance calculation unit The information system according to claim 1, wherein the information system includes one or more pieces of shooting object information including the distance calculated by. The output information acquisition unit
Obtaining output information including one or more area information and one or more distances included in the shooting object information received by the receiving unit;
The output unit is
The information system according to claim 4, further comprising a distance output unit that outputs a distance included in the one or more pieces of output information acquired by the output information acquisition unit. Further comprising one or more web server devices;
The web server device
An object-related information storage unit capable of storing object-related information that is information about the object;
An object-related information acquisition unit that acquires object-related information in response to an instruction from the outside;
An object related information transmitting unit for transmitting the object related information,
The terminal device
An object-related information receiving unit for receiving object-related information;
The output unit is
The information system according to claim 1, further comprising an object related information output unit that outputs the object related information received by the object related information receiving unit. An object information storage unit capable of storing one or more object information having an object identifier for identifying an object and area information indicating an area in which the object exists;
A terminal information acquisition unit that acquires terminal information including position information indicating a position of the terminal device and direction information indicating a direction in which the terminal device is facing;
The position information included in the terminal information acquired by the terminal information acquisition unit, the direction information included in the terminal information, and the viewing angle stored in advance are applied to the area information of each object information in the object information storage unit. A shooting object determining unit that determines one or more objects that the terminal device is shooting;
A shooting object information acquisition unit that acquires, from the object information storage unit, one or more shooting object information including region information among one or more object information corresponding to each of the one or more objects determined by the shooting object determination unit; ,
A camera unit for acquiring images;
A camera video output unit that outputs video acquired by the camera unit; and an output unit that includes a region output unit that outputs the region information included in the one or more shooting object information so as to overlap the video. ,
The shooting object information determination unit
The objects are inspected in order from the shortest distance from the terminal device, and information on an area that becomes invisible by photographing the object that has been determined as being photographed by the terminal device is registered as an invisible area, and the visible area disappears. A terminal device that terminates the inspection of whether or not the object is photographed by the terminal device. The region information includes two-dimensional region information indicating a two-dimensional region on the map and height information indicating the height of the object,
The shooting object determination unit
Using the position information and direction information included in the terminal information and the viewing angle stored in advance, a shooting area in a three-dimensional space is acquired, and the shooting area is set as each object information in the object information storage unit. Applied to the area information that the terminal device has, determining one or more objects that the terminal device is photographing, and
The objects are inspected in order from the shortest distance from the terminal device, and information on an area that becomes invisible by photographing the object that has been determined as being photographed by the terminal device is registered as an invisible area, and the visible area disappears. The terminal device according to claim 7, wherein the inspection of whether or not the object is photographed by the terminal device is terminated. The terminal device which comprises the information system in any one of Claims 1-6. The server apparatus which comprises the information system in any one of Claims 1-6. On the recording medium,
Storing one or more object information having an object identifier for identifying an object and area information indicating an area in which the object exists;
An information processing method realized by a terminal information acquisition unit, a shooting object determination unit, a shooting object information acquisition unit, a camera unit, and an output unit,
A terminal information acquisition step of acquiring terminal information having position information indicating a position of the terminal device and direction information indicating a direction in which the terminal device is facing;
The position information included in the terminal information acquired in the terminal information acquisition step, the direction information included in the terminal information, and the viewing angle stored in advance or included in the terminal information by the shooting object determination unit, A shooting object determining step that applies to area information of each object information of the recording medium and determines one or more objects being shot by the terminal device;
The shooting object information acquisition unit acquires, from the recording medium, one or more shooting object information including area information among one or more object information corresponding to one or more objects determined in the shooting object determination step. A shooting object information acquisition unit,
A camera step of acquiring video by the camera unit;
An output step for outputting the video acquired in the camera step by the output unit, and outputting region information included in the one or more shooting object information on the video;
In the shooting object information determination step,
The objects are inspected in order from the shortest distance from the terminal device, and information on an area that becomes invisible by photographing the object that has been determined as being photographed by the terminal device is registered as an invisible area, and the visible area disappears. An information processing method for ending inspection of whether or not an object is photographed by the terminal device. On the recording medium,
Storing one or more object information having an object identifier for identifying an object and area information indicating an area in which the object exists;
Computer
A terminal information acquisition unit that acquires terminal information having position information indicating the position of the terminal device and direction information indicating a direction in which the terminal device is facing;
The location information of the terminal information acquired by the terminal information acquisition unit, the direction information of the terminal information, and the viewing angle stored in advance or included in the terminal information are stored in each object information area of the recording medium. A shooting object determination unit that applies to information and determines one or more objects that the terminal device is shooting;
A shooting object information acquisition unit that acquires, from the recording medium, one or more shooting object information including region information among one or more object information corresponding to each of the one or more objects determined by the shooting object determination unit;
A camera unit for acquiring images;
A program for outputting a video acquired by the camera unit and superimposing on the video to function as an output unit that outputs area information included in the one or more shooting object information,
The shooting object information determination unit;
The objects are inspected in order from the shortest distance from the terminal device, and information on an area that becomes invisible by photographing the object that has been determined as being photographed by the terminal device is registered as an invisible area, and the visible area disappears. A program for causing the terminal device to function as a test to end whether or not the object is photographed by the terminal device.

Images