JP2008219390A - Image reader - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image reader displaying a photographed image linked with a map image. <P>SOLUTION: The image reader has a storage device 102 registering a plurality of photographed images with meta data containing each positional information of a camera and a photographing object and a display means 103 displaying the map image having the positional information and the photographed images. The image reader further has a conversion means 107 conducting a projection conversion on the basis of turning information to turn the map image in a three-dimensional space and an eye-direction computing means 10 computing the eye direction from the place of a preset virtual viewer to the place of the photographing object by using each of the projection-converted map image and the positional information of the photographing object. The image reader further has an image selection means 11 computing the eye directions from the places of the camera in a plurality of the photographed images to the photographing object and selecting images having smallest crossed angles in each eye direction as display images and an image-display control means 12 displaying the display images at the places of the photographing object on the map images. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image browsing apparatus, and more particularly, to an image selection technique and an interface for selectively browsing a plurality of captured images stored in a computer.

  In recent years, consumer photography devices such as digital still cameras and digital video cameras have become smaller and cheaper, and the built-in camera of a mobile phone has a performance that can withstand practical use. However, it is not uncommon to always carry a photo shooter.

  In addition, since digital photos can be easily erased, it is easy to shoot, and because of the sudden drop in prices of recording devices such as hard disks, unorganized image data recorded on hard disks can be personalized. It is no longer unusual to have a large number.

  In addition to reducing the size and cost of the equipment, it also distributes image data, such as exchanging captured digital images using the e-mail function of mobile phones and uploading and downloading them through networks such as the Internet. The improvement of the environment to increase the value of possessing a large amount of image data by an individual.

  Commercially, efforts are being made to collect image data taken by individuals for the purpose of providing easy-to-understand information with print services and images.

  On the other hand, with the widespread use of GPS in in-vehicle navigation systems, receivers that are small enough to be carried by people have also been developed, and it is possible to easily acquire accurate position information of people It is like that.

  A method has been proposed in which the position information is added to image data as metadata to further increase the utility value of the captured image (see, for example, Patent Document 1). This is largely due to the fact that geographical information by maps and satellite photographs is open to the public and an environment in which it can be used relatively easily has been established. The reason for this is that the release of geographic information is expected to be established commercially in the same way as Internet text search.

  As an example, Patent Document 1 discloses a method of associating a captured image with a map image included in geographic information using metadata assigned to the captured image and organizing the image based on the map image. This is a method for managing image data of a digital photograph by a computer instead of managing a photograph of a film camera by an album.

As yet another example, Patent Document 2 discloses a method that makes it easier to understand the shooting situation by registering the shooting direction together with the shooting position.
JP 2001-189905 A JP-A 2004-118685

  As in the invention described in Patent Document 1, the method of managing and browsing images by linking a map or satellite photograph and a captured image using the positional information of the captured object as a clue, provides various expressions for management browsing. It is possible not only to facilitate retrieval of stored and stored images but also to provide enjoyment of browsing.

  However, when the number of images taken at or near the same location increases, there arises a problem that the registered images are sequentially viewed as in the case of finally viewing a film photo album.

  In addition, since the captured object is not always near the shooting position, if the viewer and the photographer are not the same person, the shooting location is known, but it is impossible to determine where the shooting target is located. is there. In this case, the invention described in Patent Document 2 provides a certain direction by providing the shooting direction as information, but it is not a fundamental solution. That is, there are cases where it is easier to understand if the image is registered at a position where the object to be photographed is located, not at the photographing location. However, the appearance of the object is often different depending on the shooting direction, and there remains a problem that sufficient information on the object cannot be given to the viewer only by the shooting position or the position of the shooting object.

  The present invention inputs information such as the viewing direction of the viewer so that the viewer can sufficiently acquire information of the shooting target from a plurality of captured images stored in the computer, and the shooting position and the shooting target Provided is an image browsing apparatus capable of selecting and browsing a photographed image having a correlation with an actual photographing object using a direction obtained from a position.

  According to the present invention, (a) a storage device that registers a plurality of captured images to which metadata including position information at the time of shooting of the shooting device and position information of the shooting target is added, which is set in advance. (B) display means for displaying a map image having position information and the photographed image on a display screen; and (c) receiving rotation information for rotating the map image in a three-dimensional space from the outside. Conversion means for performing projective transformation for displaying the map image on the display screen based on rotation information; (d) position information of the map image projectively transformed by the conversion means; and position of the object to be imaged A first gaze direction that is a gaze direction from the preset position, which is the virtual position of the viewer on the projective transformed map image, to the position of the object to be photographed using the information. Gaze direction calculation hand to calculate (E) in the plurality of captured images registered in the storage device, a second line-of-sight direction, which is a line-of-sight direction from the position at the time of photographing by the photographing apparatus to the position of the photographing object, Image selection means for calculating each using data, and selecting an image having the smallest angle at which the first line-of-sight direction and the second line-of-sight direction intersect as a display image, and (f) on the map image An image display for controlling the display means to display an image of a mark corresponding to the shooting object at the position of the shooting object and to display the selected display image outside the display area of the map image. An image browsing apparatus including a control unit is provided.

  According to the aspect of the present invention, (a) a plurality of photographed images to which metadata including position information at the time of photographing of the photographing apparatus and position information of the photographing target is added in advance are registered. (B) display means for displaying a map image having position information and the photographed image on a display screen, (c) position information of the map image, and position information of the photographing object. A first direction which is a visual line direction from any position on the frame of the map image displayed by the display means to the position of the object to be photographed, which is a virtual position of the viewer on the map image. Gaze direction calculation means for calculating the gaze direction of the image, and (d) in the gaze direction from the position at the time of photographing of the photographing device to the position of the photographing object in the plurality of photographed images registered in the storage device. A second line-of-sight direction is (E) image selection means for selecting an image having the smallest angle at which the first line-of-sight direction and the second line-of-sight direction intersect each other as a display image; An image display for controlling the display means to display an image of a mark corresponding to the shooting object at the position of the shooting object and to display the selected display image outside the display area of the map image. An image browsing apparatus including a control unit is provided.

  Further, according to the aspect of the present invention, (a) a plurality of photographed images to which metadata including position information at the time of photographing of the photographing apparatus and position information of the photographing object is added are registered. (B) display means for displaying a map image having position information and the photographed image on a display screen, (c) position information of the map image, and position information of the photographing object. In a polar coordinate system having the origin at the center of the map image, which is the virtual position of the viewer on the map image, the four angles created by the diagonal lines of the map image frame are equally divided, A gaze direction calculation that calculates a first gaze direction that is a gaze direction from any position on the intersection of a diagonal line and a straight line that is equally divided and the frame of the map image to the position of the object to be photographed as the gaze direction. Means (d) A second line-of-sight direction that is a line-of-sight direction from the position at the time of photographing of the photographing apparatus to the position of the photographing object in the plurality of photographed images registered in the apparatus is calculated using the metadata. Image selection means for selecting, as a display image, an image having the smallest angle at which each of the first line-of-sight direction and the second line-of-sight direction intersects; (e) the position of the object to be photographed on the map image; And image display control means for controlling the display means so as to display an image of a mark corresponding to the photographing object and to display the selected display image outside the display area of the map image. An apparatus is provided.

  According to the present invention, information such as the viewing direction of the viewer is input so that the viewer can sufficiently acquire information on the object to be photographed from a plurality of photographed images stored in the computer, and the photographing position and the object to be photographed are input. It is possible to provide an image browsing apparatus that can browse a photographed image that has a correlation with an actual photographing object, using information on a direction obtained from the position of the object.

  Accordingly, since a plurality of images registered in the same place can be classified and sequentially browsed based on not only the conventional position information but also the direction of the line of sight, a new image browsing method can be provided. In addition, since multiple objects at different positions viewed from the same direction can be viewed at the same time, it is possible to convey more geographical information to the viewer than before, and images that emphasize geographical information such as landscape photos It is effective as a management and browsing method.

  Furthermore, since the action of moving the line of sight and the displayed image are linked, there is also an effect that the display image is easy to understand sensuously.

  Next, first to fourth embodiments of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. The following first to fourth embodiments exemplify devices and methods for embodying the technical idea of the present invention, and the technical idea of the present invention is the arrangement of components and the like. It is not specific to the following. The technical idea of the present invention can be variously modified within the scope of the claims.

(First embodiment)
As shown in FIG. 1, the image browsing apparatus according to the first embodiment of the present invention is position information (hereinafter referred to as “photographing position”) at the time of photographing of the photographing apparatus indicating the location of the photographing position (hereinafter referred to as “photographing position”). A plurality of metadata to which the position information (hereinafter referred to as “photographing position information”) indicating the position of the photographing object (hereinafter referred to as “photographing position”) is added. Storage device 102 for registering captured images, display means 103 for displaying map images having position information and captured images registered in storage device 102, and rotation in a three-dimensional space by external instructions Using the conversion means 107 that receives the information of the map image and performs projective conversion to the display screen based on the information, the position information of the map image that has been projectively converted by the conversion means, and the position information of the photographing object A line-of-sight direction for calculating a line-of-sight direction from a preset position on the map image in a three-dimensional space corresponding to a change in the map image due to rotation to a shooting target position and determining a first line-of-sight direction after the change due to rotation The calculation means 10 and a second line-of-sight direction, which is the line-of-sight direction from the position at the time of photographing of the photographing apparatus to the position of the photographing object in the plurality of photographed images registered in the storage device 102, are added to the photographed image. The first line-of-sight direction calculated by the line-of-sight direction calculation means 10 intersects with each of the second line-of-sight directions, which are line-of-sight directions to the positions of the photographing objects in the plurality of captured images. When there is an image selection unit 11 that selects an image with the smallest angle as a display image to be imaged and a captured image registered in the storage device 102, An image display control means 12 for controlling the display means 103 so as to display an image of a mark corresponding to the object to be photographed at a position and to display the selected display image outside the display area of the map image. It is possible to use an electronic computer such as the computer 202 shown in FIG. Here, “map information” refers to attribute information such as a map image downloaded from the Internet and position information added to the map image.

  The input unit 100 exchanges information between the user 203 shown in FIG. 2 and the computer 202 on which the image browsing apparatus 1 shown in FIG. 1 is mounted. In order to receive an operation on the image browsing apparatus 1 by the user, a character input device such as the keyboard 201 shown in FIG. 2 and a pointing device such as the mouse 204 can be used.

  The arithmetic processing unit 101 is a central processing unit (CPU) or the like, and performs numerical calculation, information processing, device control of each component device included in the image browsing apparatus 1 such as the conversion unit 107, and the like. Further, the program stored in the storage device 102 is read and analyzed, and data is processed. The arithmetic processing unit 101 is electrically connected to peripheral devices such as the display unit 103, the communication unit 105, and the temporary storage unit 104 via the system bus 106 or the input unit 100, and exchanges data and programs. Further, the “browsing program” stored in the storage device 102 is executed. Here, the “browsing program” is urged to be activated by the input means 100, and registers the photographed image in the storage device 102 and executes processing for searching and displaying the registered photographed image together with the map information on the computer. It is described to make it. The “browsing program display screen” is a screen provided with functions for displaying, searching, and setting necessary map information in order to display captured images registered by the “browsing program”. At the same time when the browsing program is started, the arithmetic processing unit 101 downloads map information from the server of the map information providing site through the communication means 105 connected to the Internet, and stores it in the temporary storage device 104. The conversion unit 107 receives rotation information when rotating the map image in the three-dimensional space from the input unit 100, and thereby converts the map information including the downloaded two-dimensional image into a three-dimensional space in the three-dimensional space by projective conversion. And displayed on the display means 103. FIG. 4 shows an example of the browsing program display screen 400. The map image 401 displayed on the browsing program display screen 400 includes a landmark A409 and a landmark B410 that indicate the registered position of the captured image, and a direction marker 411 that indicates the direction of the map. However, although the number of landmarks to be displayed is two for the sake of explanation, the number of displayed registration positions of captured images is not limited to two. Here, the “landmark” represents the position of the browsing object in which the captured image stored in the storage device 102 is registered. Further, the browsing program display screen 400 of FIG. 4 includes an address input frame 402 for searching and specifying the display position of the map image 401, an input address determination button 403, a slide bar 406 for determining a map scale, and a line-of-sight movement icon 407. Then, a line-of-sight movement determination button 404 and a mode button 405 for switching the mode of the browsing program display screen are displayed. The user operates the slide bar 406, the line-of-sight movement icon 407, the line-of-sight movement determination button 404, and the like with the keyboard 201 and the mouse 204 in FIG. A cursor 408 in FIG. 4 is operated by the keyboard 201 and the mouse 204. According to the processing procedure for browsing the registered photographed image shown in FIG. 3, the browsing process is executed by the image browsing device 1 executing the browsing program by the operation of the user 203 shown in FIG. The captured image can be viewed. In addition, display area designation information such as a registered address of a captured image to be browsed is input to the address input frame 402 through the input unit 100 of FIG. 1, and the search destination address is determined by the input address determination button 403. A designated map image 401 is displayed. The displayed map image 401 can be changed to a designated scale by a slide bar 406. However, the above-described method for determining the display area displayed by the map image 401 is an example, and various forms such as a method for changing and displaying the display area by dragging the map image 401 with the mouse 204 shown in FIG. It is conceivable that the browsing method of the map image 401 is not limited to the above method. 4 shows a map image in which the viewing angle of the user 203 shown in FIG. 2 who is the viewer is perpendicular to the ground, the viewing movement icon 407 and the viewing movement determination button shown in FIG. 404 can be used to arbitrarily change the angle of the line of sight with respect to the ground to change the appearance of the map image, as in the map image 501 in which the line of sight angle is changed as shown in FIG.

  The line-of-sight direction calculation means 10 is the virtual position of the viewer on the map image subjected to the projective conversion using the position information of the map image subjected to the projective conversion by the conversion means 107 and the position information of the photographing object. The first line-of-sight direction from the preset position to the position of the browsing object is calculated. Since the first line-of-sight direction changes following the rotation and movement of the map image, the first line-of-sight direction is sequentially calculated as the map image changes.

  The image selection unit 11 is most appropriate from a plurality of captured images captured by the digital camera 200 shown in FIG. 2 and registered in the storage device 102 based on the calculation result of the gaze direction by the gaze direction calculation unit 10. Select the correct image.

  The image display control unit 12 processes operation information such as rotation, movement, enlargement, and reduction of the map image input by the input unit 100, and displays an image of a mark corresponding to the shooting target, a shot image, a map image, and the like. Control the display. The display unit 103 displays the control result.

  The storage device 102 is a storage medium that can use a magnetic disk such as a hard disk, an optical disk such as a DVD (registered trademark), and a nonvolatile semiconductor memory such as a memory card. The user 203 registers the captured image in the storage device 102 in association with the map information.

  The display means 103 can be a liquid crystal monitor or the like because the user needs to visually recognize various information.

  The temporary storage device 104 is a main storage device that directly exchanges data with the arithmetic processing unit 101, and stores map information used when a user browses images, a search keyword when browsing images, and the like.

  The communication unit 105 transfers the captured image and the shooting position information from the image shooting device such as the digital camera 200 shown in FIG. 2 to the image browsing device 1 according to the operation information input to the input unit 100 by the user 203 shown in FIG. It has a communication function using wired and wireless transmission paths for transmitting and receiving image attribute data.

  The system bus 106 is a transmission path for transferring management and control commands and data between the components of the image browsing apparatus 1 including the arithmetic processing device 101 and the storage device 102.

  The conversion means 107 is a program that calculates a three-dimensional map image that can be viewed by the viewer in consideration of the light source and shape of the two-dimensional map image and changes it three-dimensionally.

  Next, the image browsing process performed by the image browsing apparatus 1 in FIG. 1 will be described using the flowchart shown in FIG.

  (A) In step S100, the arithmetic processing unit 101 shown in FIG. 1 executes the browsing program stored in the storage device 102, and displays the browsing program display screen on the display means 103.

  (B) In step S101, the input means 100 selects a landmark to be viewed from the landmarks displayed on the map image 401 shown in FIG. 4 using the keyboard 201 and the mouse 204 shown in FIG. For example, a landmark is selected from the two registered landmarks A409 and B410 on the map image 401 displayed in FIG. One or a plurality of landmarks can be selected. In this embodiment, the following description will be made assuming that both the landmark A409 and the landmark B410 are selected. Also, the map image 401 in FIG. 4 is rotated and displayed by the converting means 107 as 401 in FIG.

  (C) In step S102, the input unit 100 switches the display mode of the browsing program display screen 500 shown in FIG. 5 to the browsing program display screen 600 shown in FIG. As shown in FIG. 6, the landmarks registered in the landmark A409 and the landmark B410 selected in step S101 from the plurality of captured images stored in the storage device 102 on the screen after switching, respectively. An A image 801 and a landmark B image 601 are selected and displayed. Details of the selection method will be described later.

  (D) When viewing another landmark by changing the viewing area in step S103, the processing of the viewing program controlled by the arithmetic processing unit 101 proceeds to the processing in step S100. By inputting a new browsing area in step S100, a new browsing area is designated. At this time, the browsing program display screen 600 can be switched to the browsing program display screen 500 of FIG. If no new viewing area is designated, the program ends. The program is terminated by an end button 412 shown in FIG.

  Note that the conventional image browsing apparatus does not have the line-of-sight movement icon 407, the line-of-sight movement determination button 404, and the mode button 405 shown in the browsing program display screen 400 of FIG. 4 as shown in the browsing program display screen 700 of FIG. Other than that, the image browsing apparatus 1 is the same as the image browsing apparatus 1 of the present invention. Therefore, the conventional image browsing apparatus is the same as the image browsing apparatus 1 of the present invention in operations up to the landmark selection in step S101 except that the map image 401 cannot be changed by projective transformation. However, in the conventional image browsing apparatus, when a landmark is selected, a list of captured images registered in the selected landmark is displayed as in a registered image list display screen 800 shown in FIG. FIG. 8 shows a state in which four photographed images 801 to 804 registered in the landmark A409 are displayed as a list. In this case, it can be seen that the object to be photographed is at the position of the landmark A409, but it is not known from which direction the photographed image has been photographed.

  Details of the rotation processing of the map image 401 in FIG. 4 will be described with reference to the conceptual diagram in FIG. In FIG. 9A, the map image 401 in FIG. 4 is placed so that the origin of the three-dimensional space composed of three orthogonal x, y, and z axes coincides with the center position of the map. At this time, when the cursor 408 shown in FIG. 4 is dragged along the direction A900 on the line-of-sight movement icon 407 shown in FIG. 9B, the map image 401 is centered on the x-axis as shown in FIG. 9A. Rotation A910 is performed. Similarly, when the cursor 408 shown in FIG. 4 is dragged along the direction B901 shown in FIG. 9B, a rotation B911 around the y-axis is performed as shown in FIG. When the cursor 408 shown in FIG. 4 is dragged along, the rotation C912 about the z-axis is performed as shown in FIG. A map image 501 shown in FIG. 5 represents an example of a map image after arbitrary rotation processing is performed by the user.

  In addition, a method for selecting a photographed image registered for viewing and displayed for browsing will be described with reference to FIGS. 11 shows that when the origin 1105 coincides with the landmark A409 in FIG. 10, the respective photographing positions of the photographed images 801 to 804 in FIG. 8 registered in the storage device 102 shown in FIG. 1 are the photographing positions in FIG. 1101 to 1104. Therefore, the direction from the shooting positions 1101 to 1104 shown in FIG. 11 toward the origin 1105 is the shooting direction. If the line-of-sight position 1100 in FIG. 11 is the user's line-of-sight origin 1000 shown in FIG. 10, the vector 1106 shown in FIG. 11 indicates the first line-of-sight direction from the user to the landmark A409 as shown in FIG. The vector 1001 is obtained. The angle formed by the vector 1106 shown in FIG. 11 which is the first viewing direction of the user with respect to the landmark A409 and the vector 1107 showing the shooting direction is another vector indicating the shooting direction which is the vector 1106 and the second viewing direction. 8 forms the minimum angle 1108 compared to the angle formed by the image, the captured image 801 illustrated in FIG. 8 captured from the capturing position 1101 that is the second viewing direction constituting the vector 1107 is the viewing origin 1000 illustrated in FIG. Is the image closest to the image from the first viewing direction. FIG. 12 is a schematic diagram of the data structure of an image registered in the storage device 102 of FIG. The shooting position on the map image 401 shown in FIG. 10 or the like is uniquely determined by the metadata in which the shooting time and the shooting position are recorded. Further, since the shooting target positions are the landmark A409 and the landmark B410 on the map image 501, the shooting direction, that is, the browsing target object from the user, from the shooting position shown in FIG. 12 and the shooting target position of the browsing target object shown in FIG. The line-of-sight information up to the landmarks A and B can be calculated. Therefore, the captured image that coincides with the first viewing direction from the viewing origin 1000 to the landmark A409 shown in FIG. 10 is registered in the storage device 102 shown in FIG. 1 as the landmark A409. A captured image 801 that is closest to the image from the first line-of-sight direction at the line-of-sight origin 1000 shown in FIG. Similarly, the captured image registered in the storage device 102 shown in FIG. 1 as the landmark B410 in FIG. 10 is also used by using the vector 1002 shown in FIG. 10 and the imaging position of the metadata added to the captured image. A captured image captured from the direction closest to the first line-of-sight direction is displayed.

  Next, using the flowchart shown in FIG. 3B, the image browsing apparatus 1 in FIG. 1 selects an appropriate image from a plurality of captured images registered in the landmark in step S102 in FIG. A method of selecting is described.

  (A) In step S200, the input unit 100 uses the keyboard 201 and the mouse 204 shown in FIG. 2 to rotate the map image 401 shown in FIG. The rotation process is performed in conjunction with the user's operation of the line-of-sight movement icon 407 in FIG. An example of the gaze information calculation will be described with reference to FIG. As illustrated in FIG. 10, when the user's position is fixed to the line-of-sight origin 1000 on the display frame 1003, the user's position on the map is relatively changed by rotating the map image 501.

  (B) In step S201, the conversion unit 107 performs projective transformation on the map image 401 in FIG. 4 that has been subjected to the rotation process in step S200 to produce a visual effect as a three-dimensional image. A map image 501 shown in FIG. In addition, the conversion means 107 shown in FIG. 1 performs projection conversion to the browsing program display screen based on the rotation information for rotating the map image from the outside in the three-dimensional space. Here, “projective transformation” is a special coordinate transformation used to display a three-dimensional object on a plane. It converts an image seen from the front into an image seen from another angle. is there. For example, a three-dimensional picture is projected on a plane photograph, or a conversion is made such that a light source is fixed and an object shadow is associated.

  (C) In step S <b> 202, the line-of-sight direction calculation means 10 included in the arithmetic processing apparatus 101 calculates “line-of-sight information” for a landmark to be viewed. Here, the “line-of-sight information” means the size from the fixed arbitrary position to the landmarks A and B that are the objects to be viewed, the first line-of-sight direction with respect to the direction marker 411 shown in FIG. It is the relative position information represented. The line-of-sight vector A1001 from the line-of-sight origin 1000 to the landmark A409, which is the position of the user changed in step S201, becomes line-of-sight information from the user to the landmark A409, and the line-of-sight vector B1002 from the line-of-sight origin 1000 to the landmark B410. Is obtained as the line-of-sight information from the user to the landmark B410.

  (D) In step S <b> 203, the image selection unit 11 included in the arithmetic processing unit 101 displays a captured image that is close to the actual state that can be visually recognized when viewing the landmarks A and B from the line-of-sight origin 1000 shown in FIG. 10. A and B are selected from a plurality of images registered in the storage device 102 in FIG. 1 and displayed on the display means 103.

  (E) When viewing another captured image by changing the line-of-sight direction from the user to the landmarks A and B shown in FIG. 10 in step S204, the processing of the viewing program controlled by the arithmetic processing unit 101 is as follows: The process proceeds to step S200. Subsequently, when there is no change in the viewing direction to the landmarks A and B shown in FIG. 10, the processing of the browsing program controlled by the arithmetic processing unit 101 ends the display of the captured image registered in the landmark. To do.

  Therefore, the user can browse the captured image while considering the direction of the line of sight to the browsing object by comparing the map image 501 shown in FIG. 6 with the captured image 801 and the landmark B image 601. . As a result, the image browsing apparatus 1 according to the present invention can provide more geographic information to the user in that the captured image can be browsed for each line-of-sight direction as compared with the conventional browsing method.

  As described above, according to the image browsing apparatus 1 according to the first embodiment, a plurality of captured images can be registered in the landmark to be browsed, and the map image on the browsing program display screen is displayed in FIG. When the display unit 103 arbitrarily displays the image, the user 203 shown in FIG. 2 displays an image whose shooting direction is closest to the shooting target in the direction of the image when the landmark on the map image is viewed through the display unit 103. It is possible to automatically select and display from a plurality of registered captured images. Furthermore, when the map image displayed on the image browsing device 1 is changed in the three-dimensional space with respect to the first line-of-sight direction of the user, a plurality of captured images registered in the landmarks to be browsed also follow. Thus, the captured image closest to the image that can be viewed from the first viewing direction of the user can be displayed in real time.

(Second Embodiment)
In the first embodiment, the image browsing apparatus according to the second embodiment of the present invention is the size of a vector from the user's line-of-sight position 1100 shown in FIG. 11 to the origin 1105 that is the position of the landmark A409. In other words, a restriction is placed on the distance from the user's viewing position to the object to be browsed, and even if the landmark is selected by the user for viewing, it is not The captured image registered in the mark can be hidden.

  As described above, according to the image browsing apparatus 1 according to the second embodiment, a predetermined condition is set for the distance from the browsing position to the browsing object, and the captured image registered in the storage device 102 illustrated in FIG. By not displaying, it is possible to provide a sense of distance in addition to the sense of line of sight from the browsing user to the object. When the position of the user's viewpoint relative to the landmark that is the object to be browsed changes due to the rotation and movement of the map image, in addition to the image showing the landmark changing, the image itself is displayed and hidden. As a result, the sensory approximation to the actual map information that humans acquire visually is further improved.

(Third embodiment)
The image browsing apparatus according to the third embodiment of the present invention can be moved along the frame of the map image 401 shown in FIG. 14 instead of rotating the map image by the line-of-sight movement icon 407 shown in FIG. A method of designating the first and second line-of-sight directions using the line-of-sight marker 1401 is used.

  Thus, according to the image browsing apparatus 1 according to the third embodiment, the line-of-sight marker 1401 that is the viewer's line-of-sight origin moves on the frame of the map image 401 shown in FIG. The relationship between the origin and the landmark is represented by a two-dimensional parameter. Therefore, the image browsing apparatus 1 according to the third embodiment rotates the conversion means 107 that changes the two-dimensional image shown in FIG. 1 to produce a three-dimensional effect by projective transformation, and rotates the map image in the three-dimensional space. The line-of-sight movement icon 407 shown in FIG.

(Fourth embodiment)
The image browsing apparatus according to the fourth embodiment of the present invention uses a polar coordinate system in which the moving position of the line-of-sight marker 1401 shown in FIG. 14 is the origin of the intersection of diagonal lines in the map image 401 shown in FIG. When a broken line that equally divides the four angles formed by the diagonal lines of the frame is drawn, the first and second line-of-sight markers 1501 that can move only at the intersections 1502 to 1509 of the diagonal lines and the broken line and the frame of the map image 401 are used. Use a method to specify the line-of-sight direction.

  As described above, according to the image browsing apparatus 1 according to the fourth embodiment, since the line-of-sight origin is limited to eight places, the calculation of the first and second line-of-sight directions can be simplified. .

(Fifth embodiment)
The image browsing apparatus according to the fifth embodiment of the present invention is the method according to claims 1 to 4, wherein the map image 401 in FIG. 4 is artificially downloaded as computer graphics, such as computer graphics, from the map information providing site server. In addition to a map image such as a bitmap image drawn in the above, satellite images and aerial photographs that are actual images are used. At this time, each image can be displayed alone, or for example, a map image can be transparently combined with a satellite photograph image and displayed.

  As described above, according to the image browsing apparatus 1 according to the fifth embodiment, an artificially drawn map image such as computer graphics and a live-action image such as a satellite photograph and an aerial photograph are switched and displayed. Alternatively, by making it possible for the user to select display by performing synthesis or the like, it contributes to improving the degree of recognition of map information for the user.

(Other embodiments)
As described above, the first to fifth embodiments of the present invention have been described. However, it should not be understood that the description and drawings constituting a part of this disclosure limit the present invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art. For example, the browsing program display screen 600 shown in FIG. 6 after the mode change of the browsing program display screen 500 shown in FIG. 5 is registered as the landmark A image 801 and the landmark B image 601 in the storage device 102 shown in FIG. The captured images are displayed side by side on the map image 501, but can also be displayed as balloons on the map image 1301 as in the browsing program display screen 1300 shown in FIG. At this time, as the map image 1301 rotates, the captured image displayed in the balloon also changes. Further, in the first to fifth embodiments of the present invention, the browsing program and the captured registered image are provided in a single computer 202, but are provided on a server connected via a network and read out when necessary. A server / client system is also available. Furthermore, the captured image stored in the storage device 102 shown in FIG. 1 is not limited to one, and a plurality of people can register and publish the same landmark.

  As described above, the present invention naturally includes various embodiments not described herein. Therefore, the technical scope of the present invention is defined only by the invention specifying matters according to the scope of claims reasonable from the above description.

It is a block diagram of the image browsing apparatus which concerns on the 1st-5th embodiment of this invention. It is a general-view figure of the utilization form of the image browsing apparatus which concerns on the 1st-5th embodiment of this invention. It is a flowchart in the browsing process of the image browsing apparatus which concerns on the 1st-5th embodiment of this invention. In the image browsing apparatus according to the first embodiment of the present invention, the GUI prompts browsing processing. In the image browsing apparatus according to the first embodiment of the present invention, the GUI prompts a browsing process when the line-of-sight direction of the map image shown in FIG. 4 is changed. In the image browsing apparatus according to the first embodiment of the present invention, it is an example of a GUI for prompting a browsing process that simultaneously displays captured images registered as browsing objects. In a conventional image browsing apparatus, the GUI prompts browsing processing. In the image browsing apparatus which concerns on the 1st-5th embodiment of this invention, it is a list of the image | photographed image registered as a browsing target object. It is a figure explaining the rotation process of a map image in the image browsing apparatus which concerns on the 1st Embodiment of this invention. In the image browsing apparatus which concerns on the 1st Embodiment of this invention, it is a figure explaining a 1st gaze direction. It is a figure which shows the arithmetic processing method at the time of displaying the image | photographed image registered into the browsing target object in the image browsing apparatus which concerns on the 1st Embodiment of this invention. In the image browsing apparatus which concerns on the 1st-5th embodiment of this invention, it is a schematic diagram of the data structure of the image | photographed image. In the image browsing apparatus according to the first to fifth embodiments of the present invention, it is an example different from the display format of the GUI in FIG. 6 that prompts the browsing process that simultaneously displays captured images. In the image browsing apparatus which concerns on the 3rd Embodiment of this invention, it is a figure which shows an example of the calculation method at the time of selecting the most suitable one image from the some image | photographed image registered into the browsing target object. is there. In the image browsing apparatus according to the fourth embodiment of the present invention, the calculation method for selecting the most appropriate image from a plurality of captured images registered in the browsing object is different from FIG. It is a figure which shows an example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image browsing apparatus 100 ... Input means 101 ... Arithmetic processing apparatus 102 ... Memory | storage device 103 ... Display means 104 ... Temporary memory | storage device 105 ... Communication means 106 ... System bus 107 ... Conversion means 200 ... Digital camera 201 ... Keyboard 202 ... Computer 203 ... User 204 ... Mouse 400, 500, 600, 700, 1300 ... Browsing program display screen 401, 501, 1301 ... Map image 402 ... Address input frame 403 ... Input address decision button 404 ... Gaze movement decision button 405 ... Mode button 406 ... Slide bar 407 ... Eye movement icon 408 ... Cursor 409 ... Landmark A
410 ... Landmark B
411 ... Direction marker 412 ... End button 601 ... Landmark A image 800 ... Registered image list display screen 801 ... Landmark B image 801-804 ... Four captured images registered in landmark A409 808 ... Angle 1000 ... Line of sight Origin 1003 ... Display frame 1100 ... Gaze position 1100-1104 ... Shooting position 1105 ... Origin 1001, 1002, 1106, 1107 ... Vector 1401, 1501 ... Gaze marker 1502-1509 ... Intersection point

Claims (5)

A storage device for registering a plurality of captured images to which metadata including metadata including position information at the time of shooting of the shooting device and position information of the shooting target is set;
Display means for displaying a map image having position information and the captured image on a display screen;
Conversion means for receiving rotation information when rotating the map image in a three-dimensional space from the outside, and performing projective transformation for displaying the map image on the display screen based on the rotation information;
Using the position information of the map image projectively transformed by the converting means and the position information of the object to be photographed, a preset virtual position of the viewer on the map image subjected to the projective transformation is set. Gaze direction calculation means for calculating a first gaze direction that is a gaze direction from the position to the position of the object to be photographed;
A second line-of-sight direction that is a line-of-sight direction from the position at the time of photographing of the photographing apparatus to the position of the photographing object in the plurality of photographed images registered in the storage device, using the metadata. Image selecting means for calculating and selecting an image having the smallest angle at which the first line-of-sight direction and the second line-of-sight direction intersect each other as a display image;
The display means displays an image of a mark corresponding to the photographing object at the position of the photographing object on the map image, and displays the selected display image outside the display area of the map image. Image display control means for controlling
An image browsing apparatus comprising: The image browsing device according to claim 1,
The image selection means is a shooting target whose distance from the preset position on the projective transformed map image to the position of the shooting target is a virtual position of the viewer exceeding a preset range. An image browsing apparatus that excludes a captured image having position information of an object from a display target. A storage device for registering a plurality of captured images to which metadata including metadata including position information at the time of shooting of the shooting device and position information of the shooting target is set;
Display means for displaying a map image having position information and the captured image on a display screen;
Using the position information of the map image and the position information of the object to be photographed, any one on the frame of the map image displayed by the display means, which is the virtual position of the viewer on the map image Gaze direction calculation means for calculating a first gaze direction that is a gaze direction from the position to the position of the object to be photographed,
A second line-of-sight direction that is a line-of-sight direction from the position at the time of photographing of the photographing apparatus to the position of the photographing object in the plurality of photographed images registered in the storage device, using the metadata. Image selecting means for calculating and selecting an image having the smallest angle at which the first line-of-sight direction and the second line-of-sight direction intersect each other as a display image;
The display means displays an image of a mark corresponding to the photographing object at the position of the photographing object on the map image, and displays the selected display image outside the display area of the map image. Image display control means for controlling
An image browsing apparatus comprising: A storage device for registering a plurality of captured images to which metadata including metadata including position information at the time of shooting of the shooting device and position information of the shooting target is set;
Display means for displaying a map image having position information and the captured image on a display screen;
In the polar coordinate system with the origin of the center of the map image, which is the virtual position of the viewer on the map image, using the position information of the map image and the position information of the photographing object, the map The four angles created by the diagonal lines of the image frame are equally divided, and the line-of-sight direction from any position on the intersection of the diagonal line and the straight line forming the equal division and the frame of the map image to the position of the object to be photographed Gaze direction calculation means for calculating a first gaze direction as the gaze direction;
A second line-of-sight direction that is a line-of-sight direction from the position at the time of photographing of the photographing apparatus to the position of the photographing object in the plurality of photographed images registered in the storage device, using the metadata. Image selecting means for calculating and selecting an image having the smallest angle at which the first line-of-sight direction and the second line-of-sight direction intersect each other as a display image;
The display means displays an image of a mark corresponding to the photographing object at the position of the photographing object on the map image, and displays the selected display image outside the display area of the map image. Image display control means for controlling
An image browsing apparatus comprising: The image browsing apparatus according to any one of claims 1 to 4,
An image characterized in that the image display control means displays a bitmap image, a satellite photograph, an aerial photograph or a composite image by any combination of the bitmap image, the satellite photograph and a plurality of aerial photographs as the map image. Browsing device.

Images