JP2018185182A - Position specifying device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately specify the position and bearing of an observation object in a specific space such as the inside of a building.SOLUTION: A position specifying device 100 includes a data processing control unit 1 for specifying, on the basis of reference data 20 that includes position information 201 indicating a specific position in an observation object space, bearing information 202 indicating bearing and an image 203 imaged facing a prescribed bearing at the specific position and observation object data 321 that includes an image 322 imaged by an imaging device 31 possessed by the observation object, the position of the observation object and the bearing of the observation object in the observation object space. The data processing control unit compares the image included in the reference data with the image included in the observation object data, and specifies the position and bearing of the observation object on the basis of the position information and bearing information of the reference data that includes an image that matches or resembles the image included in the observation object data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a position specifying device that specifies the position and orientation of an observation target in a target space.

  In a position information service using a mobile terminal such as a smartphone, a technique is known in which the position of an observation target is specified using GPS (global positioning system) (see, for example, Patent Document 1).

JP 2009-63336 A

  In recent years, there is a need to improve productivity by analyzing the movement of people in limited spaces such as offices, restaurants and factories. For example, in a limited space such as a kitchen in a restaurant, by analyzing the trajectory of the movement of the employee as the observation target, the employee's behavior can be improved and the layout in the kitchen can be improved. By doing so, it becomes possible to improve productivity.

  However, in indoor locations such as underground malls and buildings, the positioning accuracy of GPS receivers built into mobile terminals such as smartphones decreases, so the position of the observation subject who owns the mobile terminal must be accurately identified. It is difficult.

  As one method for solving this problem, by placing wireless devices such as beacons at regular intervals in the positioning target space, and receiving radio waves emitted from those wireless devices with a mobile terminal such as a smartphone, A method for specifying the position of the portable terminal in the space to be measured can be considered. However, with this method, the member cost and installation cost of a wireless device and the like cannot be ignored, resulting in an increase in cost. In addition, when the radio wave of the wireless device is disturbed, there is a possibility that the exact position of the observation subject cannot be specified.

  Moreover, with the above-described method using GPS or beacon, it is difficult to accurately specify which direction (orientation) the observation target person is facing. For example, in the case of a kitchen in which a counter and a cooking stove are placed facing each other across the passage, whether the employee in the passage is facing the counter for customer service, or the cooking stove on the opposite side of the counter It is difficult to accurately determine whether or not cooking is done.

  The present invention has been made in view of the above-described problems, and an object thereof is to more accurately specify the position and orientation of an observation target in a specific space.

  A position specifying device according to a representative embodiment of the present invention includes position information indicating a specific position in a positioning target space, direction information indicating an azimuth at the specific position, and a predetermined direction at the specific position. A storage unit that stores reference data in association with an image captured facing the camera, observation target data that includes an image captured by an imaging device included in the observation target, and the reference data stored in the storage unit. Based on a data processing control unit that specifies the position of the observation target and the direction of the observation target in the positioning target space, the data processing control unit, the data acquisition unit for acquiring the observation target data; An image collation unit that collates an image included in the observation target data acquired by the data acquisition unit with an image included in the reference data, and for verification by the image verification unit Based on the position information and the azimuth information of the reference data including an image determined to match or similar to the image included in the observation target data, a specifying unit that specifies the position and orientation of the observation target; It is characterized by including.

  According to the position specifying apparatus according to the present invention, the position and orientation of the observation target in a specific space can be specified more accurately.

It is a figure which shows the structure of the position specific system containing the position specific apparatus which concerns on Embodiment 1. FIG. It is a figure which shows an example of the imaging | photography method using a portable terminal. It is a figure which shows an example of the image image | photographed with the portable terminal. It is a figure which shows an example of a positioning object space. It is a figure for demonstrating the azimuth | direction in the positioning object space. 6 is a flowchart showing a flow of processing for specifying the position and orientation of an observation target in a positioning target space by the position specifying device according to Embodiment 1. FIG. It is a figure which shows the structure of the position specific system containing the position specific apparatus which concerns on Embodiment 2. FIG. It is a figure which shows an example of a magnetic field map. It is a figure for demonstrating the identification method of the direction by a magnetic field collation part. FIG. 10 is a flowchart showing a flow of processing for specifying the position and orientation of an observation target in a positioning target space by the position specifying device according to the second embodiment.

1. First, an outline of a typical embodiment of the invention disclosed in the present application will be described. In the following description, as an example, reference numerals on the drawings corresponding to constituent elements of the invention are shown in parentheses.

  [1] The position specifying device (100, 100A) according to the representative embodiment of the present invention includes position information (201) indicating a specific position (50_1 to 50_n) in the positioning target space (5), and the specifying A storage unit (2, 2A) that stores reference data (20) that associates the azimuth information (202) indicating the azimuth at the position and the image (203) photographed with the specific position facing the predetermined azimuth. ), Observation target data (321, 321A) including an image (322) taken by the imaging device (31) of the observation target (4), and the reference data stored in the storage unit, A data processing control unit (1, 1A) for specifying the position of the observation target and the direction of the observation target in the positioning target space, wherein the data processing control unit is a data acquisition unit for acquiring the observation target data. An acquisition unit (11, 11A), an image verification unit (12, 12A) for verifying an image included in the observation target data acquired by the data acquisition unit and an image included in the reference data, and the image verification A specifying unit that specifies the position and orientation of the observation target based on the position information and the azimuth information of the reference data including an image determined to match or similar to an image included in the observation target data (13, 13A).

  [2] In the position specifying device (100A), the storage unit (2A) further stores a magnetic field map (22) including information on the magnetic field in the positioning target space, and the observation target data is transmitted by the imaging device. It further includes magnetic field information (323) at the position where the image is taken, and the data processing control unit further includes a magnetic field collating unit (14) for collating the magnetic field map with the magnetic field information included in the observation target data. The specifying unit (13A) may specify the position and orientation of the observation target in the positioning target space based on the matching result by the image matching unit and the matching result by the magnetic field matching unit.

  [3] In the position specifying device (100A), the observation target data includes a first vector (B) indicating the magnitude and direction of a magnetic field measured when an image is captured by the imaging device, and the magnetic field The map includes a second vector (B1 to Bn) indicating the magnitude and direction of the magnetic field measured at the specific position in the positioning target space, and the magnetic field measured in a predetermined direction in the positioning target space. A third vector (B0) indicating a magnitude and a direction, the magnetic field collating unit collates the first vector and the second vector, determines a position candidate based on the collation result, A direction candidate may be determined based on the second vector, the first vector, and the third vector at the candidate position.

  [4] In the position specifying device (100A), the image matching unit determines position and orientation candidates based on the reference data of an image that matches or is similar to an image included in the observation target data, The specifying unit selects a position and orientation closest to the position and orientation candidate determined by the magnetic field verification unit from the position and orientation candidates determined by the image verification unit, and the selected position and orientation are The position and orientation of the observation target in the positioning target space may be used.

  [5] A position specifying system (500, 500A) according to a typical embodiment of the present invention includes the position specifying device and a portable terminal (3, 3A) having the imaging device, The position specifying device is connected by wire or wireless.

2. Specific Examples of Embodiments Hereinafter, specific examples of embodiments of the present invention will be described with reference to the drawings. In the following description, the same reference numerals are given to components common to the respective embodiments, and repeated description is omitted.

<< Embodiment 1 >>
FIG. 1 is a diagram showing a configuration of a position specifying system including a position specifying device according to Embodiment 1 of the present invention.
A position specifying system 500 shown in the figure is a system for specifying the position and orientation of an observation target (for example, a person) in a positioning target space, and includes a position specifying device 100 and a mobile terminal 3.

  The position specifying device 100 is a device for specifying the position and orientation of the observation target in the positioning target space from the image taken by the mobile terminal 3 possessed by the observation target (for example, a person) in the positioning target space.

  Here, the positioning target space is a space indicating a range in which the position can be specified, and is, for example, a limited space such as one floor of an office or a site such as a factory, a restaurant or a kitchen. In the present embodiment, a case where the positioning target space is indoor will be described as an example.

  The mobile terminal 3 is a device for taking an image of the surrounding environment at the position where the observation target exists, and can be attached to, for example, a small electronic device typified by a smartphone with a built-in camera, a human clothes, A wearable terminal with a built-in camera.

Specifically, the mobile terminal 3 includes an imaging device 31, a storage unit 32, and an output unit 33.
The imaging device 31 is a functional unit for taking an image, and is, for example, a camera configured to include an optical system such as an imaging element and a lens. The imaging device 31 may have a configuration capable of shooting not only a still image but also a moving image.

  The storage unit 32 is a functional unit that stores an image 322 taken by the imaging device 31, and includes a nonvolatile memory such as a flash memory, for example. The output unit 33 is a functional unit for outputting an image stored in the storage unit 32 to an external device, and includes an external interface circuit for transmitting / receiving data to / from the external device by wire or wireless.

FIG. 2 is a diagram illustrating an example of a photographing method using the mobile terminal 3.
As shown in FIG. 2, when an image is captured using the mobile terminal 3 in the positioning target space 5, so that the direction (front direction) in which the observation target 4 having the mobile terminal 3 is facing can be captured. It is desirable to adjust the orientation of the imaging device 31 in advance.

FIG. 3 is a diagram illustrating an example of an image captured by the mobile terminal 3.
An image 322 taken by the imaging device 31 of the portable terminal 3 as shown in the figure is stored in the storage unit 32 as observation target data 321 together with information accompanying the image 322 such as information indicating the time of taking. The data can be read out via the output unit 33.

  The imaging device 31 performs imaging every unit time, and sequentially stores the captured images 322 in the storage unit 32. For example, the imaging device 31 captures a still image every unit time (for example, 1 minute), and sequentially stores the still image as an image 322 in the storage unit 32. Alternatively, the imaging device 31 may capture a moving image, cut out a still image from the moving image every unit time, and sequentially store the cut-out still image as the image 322 in the storage unit 32.

The position specifying device 100 includes a data processing control unit 1 and a storage unit 2.
The storage unit 2 is a functional unit for storing reference data 20 serving as a reference for specifying the position and orientation of the observation target in the positioning target space based on the image taken by the mobile terminal 3 included in the observation target. .

  The storage unit 2 is, for example, an external storage device such as an HDD (Hard Disc Drive) or a flash memory that can be connected to a personal computer or the like, a data server, and various types of computers built in the data processing control unit 1 described later. It can be realized by a storage device or the like.

Here, the reference data 20 will be described in detail.
The reference data 20 includes position information 201 indicating a specific position in the positioning target space 5, azimuth information 202 indicating an azimuth at the specific position, and an image 203 photographed at a specific position and facing a predetermined azimuth. The associated data.

FIG. 4 is a diagram illustrating an example of a positioning target space.
As shown in the figure, the positioning target space 5 is represented by a region on the XY plane formed by the X axis and the Y axis orthogonal to the X axis. Specific positions 50_1 to 50_n (n is an integer of 2 or more) in the positioning target space 5 are represented by coordinates on the X axis and the Y axis. Therefore, the position information 201 indicating the specific positions 50_1 to 50_n of the positioning target space 5 is information on the X and Y coordinates on the XY plane representing the positioning target space 5.

FIG. 5 is a diagram for explaining the orientation in the positioning target space.
As shown in the figure, a plurality of azimuths are specified for each specific position 50 — i (1 ≦ i ≦ n) in the positioning target space 5. For example, as shown in FIG. 3, at a specific position 50_i, north (N), northeast (NE), east (E), southeast (SE), south (S), southwest (SW), west (W) , And north-west (NW) eight directions. In this example, the orientation information 202 includes north (N), northeast (NE), east (E), southeast (SE), south (S), southwest (SW), west (W), and northwest (NW). It becomes the information which shows either direction.

  The image 203 is an image taken in a predetermined direction at a specific position. For example, the image 203 is an image taken in the above eight directions at each position 50_1 to 50_n in the positioning target space 5 shown in FIG.

  As described above, the image 203 photographed in each direction at the specific positions 50_1 to 50_n in the positioning target space 5 is stored as the reference data 20 together with the position information 201 and the direction information 202 when the image 203 is photographed. Stored in part 2.

  The reference data 20 is stored in the storage unit 2 in advance before the position specifying process by the position specifying device 100 is executed. For example, a small survey device (robot) equipped with a camera is moved in the positioning target space 5, and images 203 in each direction are photographed by the camera at specific positions 50_1 to 50_n. Then, the captured images 203 may be stored in the storage unit 2 as the reference data 20 in association with the position information 201 and the orientation information 202 when the images 203 are captured.

  As a camera used when creating the reference data 20, for example, an omnidirectional camera (so-called 360 degree camera) capable of capturing images in all directions, up and down, left and right is preferable. According to this, by specifying one azimuth (for example, “north”) in a 360-degree panoramic image photographed at a specific position 50_i, a known image processing technique can be used to set each azimuth as a reference. It becomes easy to cut out the image from the panoramic image, and it is possible to efficiently acquire images in each direction at each specific position 50_1 to 50_n in the positioning target space 5.

  Further, the height of the camera from the ground (the height in the Z direction) when the image 203 of the reference data 20 is taken is the height from the ground of the imaging device 31 when the imaging device 31 is attached to the observation target 4. It is desirable to match the height (the height in the Z direction).

Next, the data processing control unit 1 will be described.
The data processing control unit 1 determines the position and orientation of the observation target in the positioning target space 5 based on the observation target data 321 including the image taken by the mobile terminal 3 and the reference data 20 stored in the storage unit 2. It is a functional part for specifying. The data processing control unit 1 includes a data acquisition unit 11, an image collation unit 12, and a specification unit 13.

  Here, the data processing control unit 1 is realized by a computer such as a personal computer, for example. Specifically, an arithmetic device such as a CPU (Central Processing Unit), a storage device such as a memory or an HDD, and an input device that detects an input of information from the outside such as a keyboard, a mouse, a pointing device, a button, or a touch panel A computer having an I / F (Interface) device that transmits and receives information to and from the outside, and a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) is installed on the computer. By executing, the data acquisition unit 11, the image matching unit 12, and the specifying unit 13 are realized.

  The data processing control unit 1 and the mobile terminal 3 are connected by wire or wireless, and data can be transmitted and received between them.

  The data acquisition unit 11 is a functional unit for acquiring an image taken by the mobile terminal 3. The data acquisition unit 11 acquires observation target data 321 output from the output unit 33 of the mobile terminal 3 by wired communication or wireless communication. For example, when the data processing control unit 1 is a personal computer and the portable terminal 3 is compatible with USB (Universal Serial Bus) communication, the personal computer as the data processing control unit 1 and the portable terminal 3 are connected by a USB cable. By connecting, the data acquisition unit 11 reads the observation target data 321 from the storage unit 32 of the mobile terminal 3.

  The image collation unit 12 performs image collation processing for collating the image 203 included in the reference data 20 stored in the storage unit 2 with the image 322 included in the observation target data 321 acquired by the data acquisition unit 11. In this image matching process, the degree of coincidence between both images is determined based on the degree of coincidence between the feature amount extracted from the image 322 photographed by the mobile terminal 3 and the feature amount extracted from the image 203 included in each reference data 20. Data processing for determination. This image collation process can be realized by applying various well-known image matching techniques.

  When the image 203 that is closest (matched or most similar) to the image 322 included in the observation target data 321 is found by the image matching process, the image matching unit 12 determines the position 50_1 specified by the reference data 20 including the image 203. ˜50 — n and the direction are set as candidates for the position and direction of the observation target 4 in the positioning target space 5, and information on the candidates is given to the specifying unit 13 as the image matching result 120.

  The image matching result 120 includes position information 201 and azimuth information 202 included in the reference data 20 related to the image 203 that matches or is similar to the image 322 of the observation target data 321.

  The identifying unit 13 identifies the position and orientation of the observation target 4 in the positioning target space 5 based on the image matching result 120 by the image matching unit 12. Specifically, the specifying unit 13 uses the positions 50_1 to 50_n specified by the position information 201 included in the image matching result 120 as the positioning target space of the observation target 4 at the time when the image 322 of the observation target data 321 is captured. The position within 5.

  Further, the specifying unit 13 sets the direction specified by the direction information 202 included in the image matching result 120 as the direction (direction) of the observation target 4 at the time when the image 322 of the observation target data 321 is taken.

Next, the flow of the position specifying process by the position specifying device 100 will be described.
FIG. 6 is a flowchart showing a flow of position specifying processing by the position specifying device 100.

  First, in the position specifying device 100, the data acquisition unit 11 of the data processing control unit 1 acquires the observation target data 321 from the portable terminal 3 by the above-described method, and performs image verification on the image 322 included in the observation target data 321. It gives to the part 12 (step S1).

  Next, the image matching unit 12 starts the above-described image matching process based on the image 322 of the observation target data 321 acquired in step S1 and the image 203 of the reference data 20 stored in advance in the storage unit 2. (Step S2).

  In the image matching process, the image matching unit 12 determines whether or not the reference data 20 including the image 203 that matches or is similar to the image 322 of the observation target data 321 in the reference data 20 stored in the storage unit 2 ( Step S3).

  In step S3, when there is no reference data 20 including the image 203 that matches or is similar to the image 322 in the reference data 20 stored in the storage unit 2, the position specifying process is stopped.

  In step S <b> 3, when there is the reference data 20 including the image 203 that matches or is similar to the image 322 in the reference data 20 stored in the storage unit 2, the image matching unit 12 matches or most matches the image 322. The position information 201 and the azimuth information 202 included in the reference data 20 related to the similar image 203 are given to the specifying unit 13 as the image matching result 120 (step S4).

  Next, based on the image matching result 120 received in step S4, the specifying unit 13 uses the above-described method to determine the positioning target of the observation target 4 at the time when the image 322 of the observation target data 321 acquired in step S1 was taken. The position and orientation in the space 5 are specified (step S5).

  The position specifying device 100 performs the processes from steps S1 to S5 described above for each observation target data 321 stored in the storage unit 32 of the mobile terminal 3, so that an image 322 of each observation target data 321 is captured. The position and orientation of the observation target 4 in the positioning target space 5 for each time can be specified.

  As described above, the position specifying device 100 according to Embodiment 1 collates the image 322 included in the observation target data 321 captured by the mobile terminal 3 with the image 203 included in the reference data 20 for each specific position, Based on the position information 201 and the azimuth information 202 of the reference data 20 including the image 203 that matches or resembles the image 322 included in the observation target data 321, the position and orientation of the observation target to be observed are specified.

  According to this, since the position of the observation target in the positioning target space 5 and the direction of the observation target can be specified from the image taken by the mobile terminal 3 of the observation target, the positioning accuracy of the GPS receiver It is possible to accurately specify the position and orientation of the observation target even indoors such as in an underground shopping center or a building where the fall of the brightness is reduced.

  In addition, by arranging data including the position of the observation target, the direction (direction) of the observation target, and the time related to each observation target data 321 specified by the position specifying device 100 in time series, the observation target in the positioning target space 5 Trajectory data indicating the trajectory of the action can be easily generated.

  Thereby, it becomes easy to analyze the movement of a person in a limited space such as an office, a restaurant, and a factory, and productivity can be improved. For example, in the case of the restaurant kitchen described above, by analyzing the trajectory data generated based on the position and orientation of the employee specified by the position specifying device 100, the employee's action can be made more efficient, It becomes possible to improve the layout in the kitchen, and to improve productivity.

<< Embodiment 2 >>
FIG. 7 is a diagram illustrating a configuration of a position specifying system including the position specifying device according to the second embodiment.
The position specifying device 100A in the position specifying system 500A shown in the figure specifies the position and orientation of the observation target using the image taken by the mobile terminal 3A and the magnetic field information measured by the mobile terminal 3A. However, it is different from the position specifying device 100 according to the first embodiment, and is otherwise the same as the position specifying device 100 according to the first embodiment.

  The mobile terminal 3A in the second embodiment further includes a magnetic sensor 34 in addition to the functional units of the mobile terminal 3 according to the first embodiment.

  The magnetic sensor 34 measures vector information indicating the magnitude and direction of the magnetic field in the direction of three axes (X axis, Y axis, and Z axis orthogonal to each other). Specifically, the magnetic sensor 34 uses a magnetic field vector B = (x ′, y ′, z ′) as a first vector indicating the magnitude and direction of the magnetic field measured when the image 322 is captured by the imaging device 31. And stored in the storage unit 32 as the magnetic field information 323.

  The portable terminal 3A, for example, every time an image is captured by the imaging device 31, the captured image 322, the magnetic field information 323 measured by the magnetic sensor 34, the information on the time when the image 322 was captured, and the like as observation target data 321A. Store in the storage unit 32.

  In addition to the reference data 20, a magnetic field map 22 is stored in the storage unit 2 in the storage unit 2A of the position specifying device 100A according to the second embodiment. The magnetic field map 22 is data including information on the magnetic field of the positioning target space 5.

FIG. 8 is a diagram illustrating an example of a magnetic field map.
As shown in FIG. 8, the magnetic field map 22 is information on the magnetic field in the positioning target space 5 as a three-dimensional space. Specifically, the magnetic field map 22 includes a second vector and a third vector as vector information indicating the magnitude and direction of the magnetic field in the directions of three axes (X axis, Y axis, and Z axis orthogonal to each other).

Here, the second vector is a magnetic field vector B1 = (x ₁ , y ₁ , z ₁ ) to Bn = (x) indicating the magnitude and direction of the magnetic field measured at specific positions 50_1 to 50_n in the positioning target space 5. _{n, y n,} is the _{z n).}

The third vector is a reference vector B0 = (x ₀ , y ₀ , z ₀ ) indicating the magnitude and direction of the magnetic field measured in a predetermined direction in the positioning target space 5. For example, as illustrated in FIG. 8, when the true north (the direction of geomagnetism) is set as the reference direction at the position 50_1 in the positioning target space 5, the magnitude and direction of the magnetic field measured with the magnetic sensor toward the reference direction. Is a reference vector B0 = (x ₀ , y ₀ , z ₀ ).

  The place where the reference vector is measured may be any position in the positioning target space 5, and is not limited to the above-mentioned “position 50_1”. Further, the reference orientation for measuring the reference vector B0 does not need to be “true north”, and may be adjusted to a known arbitrary orientation.

The magnetic field map 22 is stored together with the reference data 20 in the storage unit 2A in advance before the position specifying process by the position specifying device 100 is executed. For example, as in the case of the reference data 20, a small survey apparatus (robot) equipped with a magnetic sensor is moved in the positioning target space 5, and the magnetic field vector B1 measured by the magnetic sensor for each specific position 50_1 to 50_n. = (X ₁ , y ₁ , z ₁ ) to Bn = (x _n , y _n , z _n ) are associated with position information indicating each specific position 50_1 to 50_n, and stored as information of the magnetic field map 22 What is necessary is just to memorize | store in 2A.

  In addition, the reference vector B0 when facing a specific direction at an arbitrary position in the positioning target space 5 is measured by a magnetic sensor mounted on the small survey device, and stored in the storage unit 2A as information of the magnetic field map 22. For example, a small survey robot is placed at a position 50_1, the magnetic sensor is directed to the true north direction (the direction of geomagnetism) at that location, and the magnitude and direction of the magnetic field are measured. B0 may be used.

When the magnetic field map 22 is created, the three axes (local coordinates) serving as the measurement reference of the magnetic sensor mounted on the small survey device and the three axes (global coordinates) representing the positioning target space 5 (map) are matched. It is necessary to keep it. For example, when the global coordinates representing the positioning target space 5 are set to the X axis, the Y axis, and the Z axis shown in FIG. 4, the X axis, the Y axis, and the measurement reference of the magnetic sensor mounted on the small survey device, The magnetic field vectors B1 to Bn at the respective positions 50_1 to 50_n in the positioning target space 5 are measured in a state where the local coordinates of the Z axis are made coincident with the X axis, the Y axis, and the Z axis in FIG.
Therefore, the magnetic field vector on the local coordinates at an arbitrary position 50 — _i is Bi ′ = (x _i ′, y _i ′, z _i ′), and the magnetic field vector on the global coordinates at the arbitrary position 50 — _i is Bi = (x _i , y _i , z _i ), the magnetic field vector Bi at an arbitrary position 50 — _{i of} the magnetic field map 22 is represented by Bi = (x _i , y _i , z _i ) = (x _i ′, y _i ′, z _i). ').

  The height of the magnetic sensor from the ground (height in the Z direction) when measuring the magnetic field vectors B1 to Bn and the reference vector B0 of the magnetic field map 22 is the ground of the magnetic sensor 34 attached to the observation target 4. It is desirable that the height be approximately the same as the height from (the height in the Z direction).

  The data processing control unit 1A of the position specifying device 100A according to the second embodiment includes a data acquisition unit 11A, an image matching unit 12A, a specifying unit 13A, and a magnetic field matching unit 14.

  Similarly to the data acquisition unit 11 according to the first embodiment, the data acquisition unit 11A acquires the observation target data 321A output from the output unit 33 of the mobile terminal 3A by wired communication or wireless communication. The data acquisition unit 11A extracts the image 322 from the observation target data 321A acquired from the mobile terminal 3A and gives the image 322 to the image verification unit 12A, and extracts the magnetic field information 323 from the observation target data 321A. give.

  The image matching unit 12A performs an image matching process for matching the image 203 included in the reference data 20 stored in the storage unit 2A and the image 322 received from the data acquisition unit 11A. Specifically, the image collation unit 12A selects an image 203 having a feature amount that matches or is similar to the feature amount of the image 322 captured by the mobile terminal 3A. Then, the positions 50_1 to 50_n and the direction corresponding to the reference data 20 including the selected image 203 are set as the positions 50_1 to 50_n and the direction candidates in the positioning target space 5 of the observation target 4, and information on the positions and directions of the candidates is used. The image matching result 120A is given to the specifying unit 13.

The magnetic field collation unit 14 performs a magnetic field collation process for collating the magnetic field map 22 stored in the storage unit 2 </ b> A and the magnetic field information 323 given from the data acquisition unit 11. Specifically, a magnetic field vector B = (x ′, y ′, z ′) measured by the magnetic sensor 34 of the mobile terminal 3A included in the magnetic field information 323 is used as a feature value, and a magnetic field that matches or is similar to this feature value. The vector Bj = (x _j ′, y _j ′, z _j ′) is selected from the magnetic field vectors B1 to Bn included in the magnetic field map 22. The position 50 — j corresponding to the selected magnetic field vector Bj is set as a position candidate of the observation target 4 in the positioning target space 5. Note that 1 ≦ j ≦ n.

The magnetic field collation unit 14 also uses the magnetic field vector Bj at the position 50 — j as a candidate by the magnetic field collation process and the magnetic field vector B = (x ′, y measured by the magnetic sensor 34 of the mobile terminal 3A included in the magnetic field information 323. Based on ', z') and the reference vector B0 = (x ₀ , y ₀ , z ₀ ) included in the magnetic field map 22, the azimuth candidate of the observation target 4 having the mobile terminal 3A is determined. Hereinafter, the direction specifying method by the magnetic field matching unit 14 will be described in detail with reference to the drawings.

FIG. 9 is a diagram for explaining a method of specifying an orientation by the magnetic field matching unit 14.
Here, as an example, it is assumed that the observation target 4 is mounted with the magnetic sensor 34 in a state where the front direction of the observation target 4 and the Y axis of the local coordinates of the magnetic sensor 34 are matched.

As shown in the figure, the magnetic field collation unit 14 first determines the magnetic field vector Bj = (x _j ′, y _j ′, z _j ′) at the position 50 — _j that is a candidate by the magnetic field collation process and the mobile terminal 3A. Based on the difference from the magnetic field vector B = (x ′, y ′, z ′) measured by the magnetic sensor 34, the angles θ and φ are calculated.

Here, the angle θ is an angle formed between the y ′ component of the magnetic field vector B and the magnetic field vector Bj. The angle φ is an angle formed by the x′-y ′ plane determined by the y ′ component and the x ′ component of the magnetic field vector B and the magnetic field vector Bj.
As described above, when the front direction of the observation target 4 is matched with the Y axis of the local coordinates of the magnetic sensor 34, the angle θ represents the horizontal direction of the observation target 4 in the positioning target space 5, and the angle φ Represents the vertical direction of the observation target 4 in the positioning target space 5 (angle relative to the x′-y ′ plane).

  Specifically, the magnetic field matching unit 14 calculates the angle θ based on the formula (1), and calculates the angle φ based on the formula (2).

  By the above-described method, the angles θ and φ with respect to the magnetic field vector Bj of the magnetic field vector B measured by the magnetic sensor 34 can be known, and therefore the angle with respect to the reference vector B0 can be calculated from the magnetic field map 22.

Specifically, the magnetic field matching unit 14 determines an orientation based on the angles θ and φ and the reference vector B0, and sets the orientation as a candidate for the orientation of the observation target 4. More specifically, the magnetic field vector _{Bj = (x j ', y} j', z j ') based on the difference between the reference vector _{B0 = (x 0, y 0} , z 0), the magnetic field relative to a reference vector B0 The angle of the vector Bj is calculated, and the angle is corrected by the angles θ and φ (for example, θ or φ is added or subtracted), and the direction corresponding to the corrected angle is set as a candidate for the direction of the observation target 4.

  Then, the magnetic field matching unit 14 specifies, as magnetic field matching results 140, information on position candidates determined by the above-described magnetic field matching and information on candidate orientations of the observation target 4 determined based on the angles θ and φ. Part 13 is given.

  The specifying unit 13A specifies the position and orientation of the observation target 4 based on the image matching result 120A by the image matching unit 12A and the magnetic field matching result 140 by the magnetic field matching unit 14.

  Specifically, the specifying unit 13A selects the position and direction (match or match) closest to the position and direction candidates included in the magnetic field verification result 140 from the candidates of the position information 201 and the direction information 202 included in the image verification result 120A. Select the most similar position and orientation). Then, the position corresponding to the position information 201 of the selected candidate is set as the position in the positioning target space 5 of the observation target 4 at the time when the image 322 of the observation target data 321 is taken, and corresponds to the direction information 202 of the selected candidate. The azimuth is the orientation of the observation target 4 in the positioning target space 5 at the time when the image 322 of the observation target data 321 is taken.

Next, the flow of position specifying processing by the position specifying device 100A according to Embodiment 2 will be described.
FIG. 10 is a flowchart showing a flow of position specifying processing by the position specifying apparatus 100A according to the second embodiment.

  First, in the position specifying device 100A, the data acquisition unit 11 of the data processing control unit 1A acquires the observation target data 321 from the portable terminal 3A by the above-described method, and the image 322 included in the observation target data 321 is subjected to image verification. The magnetic field information 323 included in the observation target data 321 is given to the magnetic field verification unit 14 while being given to the unit 12A (step S11).

  Next, the image matching unit 12A starts the above-described image matching process based on the image 322 of the observation target data 321A acquired in step S11 and the image 203 of the reference data 20 stored in advance in the storage unit 2. (Step S12).

  In the image matching process, the image matching unit 12A determines whether or not the reference data 20 including the image 203 that matches or is similar to the image 322 of the observation target data 321A in the reference data 20 stored in the storage unit 2 ( Step S13).

  In step S13, when there is no reference data 20 including the image 203 that matches or is similar to the image 322 in the reference data 20 stored in the storage unit 2, the position specifying process is stopped.

  In step S <b> 13, when there is reference data 20 including the image 203 that matches or is similar to the image 322 in the reference data 20 stored in the storage unit 2, the image matching unit 12 </ b> A matches or is similar to the image 322. The position and orientation corresponding to the reference data 20 including the image 203 to be used are candidates for positions 50_1 to 50_n and directions in the positioning target space 5 of the observation object 4, and information on the positions and orientations of the candidates is used as the image matching result 120A. It gives to the specific part 13 (step S14).

  On the other hand, the magnetic field verification unit 14 starts the above-described magnetic field verification process based on the magnetic field information 323 of the observation target data 321A acquired in step S11 and the magnetic field map 22 stored in advance in the storage unit 2 (step S15). ).

  In the magnetic field matching process, the magnetic field matching unit 14 matches or resembles the magnetic field vector B based on the magnetic field information 323 of the observation target data 321A among the magnetic field vectors B1 to Bn of the magnetic field map 22 stored in the storage unit 2. The presence / absence of the magnetic field vectors B1 to Bn is determined (step S16).

  In step S16, when there are no magnetic field vectors B1 to Bn that match or are similar to the magnetic field vector B in the magnetic field map 22, the process proceeds to step S20 described later.

  On the other hand, in step S16, when the magnetic field map 22 includes magnetic field vectors B1 to Bn that match or are similar to the magnetic field vector B, the magnetic field matching unit 14 positions 50_j corresponding to the matching or similar magnetic field vector Bj. Are determined as position candidates of the observation target 4 in the positioning target space 5 (step S17).

  Next, the magnetic field matching unit 14 observes 4 based on the magnetic field vector Bj at the position 50_j that is a candidate in step S17, the reference vector B0 included in the magnetic field map 22, and the magnetic field vector B based on the magnetic field information 323. Are determined (step S18). Specifically, as described above, the angles θ and φ are calculated from the difference between the magnetic field vector Bj and the magnetic field vector B, and based on the angles θ and φ and the reference vector B0, candidates for the orientation of the observation target 4 To decide.

  Next, the magnetic field matching unit 14 uses the information on the candidate position of the observation target 4 determined in step S17 and the information on the candidate direction of the observation target 4 determined in step S18 as the magnetic field matching result 140 to the specifying unit 13. (Step S19).

  Next, based on the image matching result 120A received in step S14 and the magnetic field matching result 140 received in step S19, the specifying unit 13 uses the method described above to obtain the image 322 of the observation target data 321 acquired in step S11. The position and orientation of the observation object 4 at the time when is photographed are identified (step S20).

  In step S16, if there are no magnetic field vectors B1 to Bn that match or are similar to the magnetic field vector B in the magnetic field map 22, the image of step S14 is obtained in step S20 by the same method as in the first embodiment. The position and orientation of the observation object 4 are specified using the collation result 120A. That is, in this case, the position and orientation of the observation object 4 are specified by using only the image without using the magnetic field information.

  The position specifying device 100A performs the processing from step S11 to S20 described above for each observation target data 321A stored in the storage unit 32 of the mobile terminal 3A, whereby an image 322 of each observation target data 321A is taken. It is possible to specify the position and orientation of the observation object 4 for each time.

  In the processing flow shown in FIG. 10, the case where the magnetic field matching process (steps S15 to S19) is executed after the image matching process (steps S12 to S14) is illustrated, but the present invention is not limited to this. For example, the image matching process (steps S12 to S14) may be performed after the magnetic field matching process (steps S15 to S19), or the magnetic field matching process (steps S15 to S19) and the image matching process (steps S12 to S14) are performed in parallel. It may be done automatically.

  As described above, the position specifying device 100A according to Embodiment 2 includes the image matching result 120A between the image 322 included in the observation target data 321A photographed by the mobile terminal 3A and the image 203 included in the reference data 20, and the magnetic field map 22. And the position and orientation of the observation target are specified based on the magnetic field comparison result 140 with the magnetic field information 323 included in the observation target data 321A.

  This makes it possible to specify the position and orientation of the observation target 4 in the positioning target space 5 more accurately than when only the image is used. In particular, when the positioning target space 5 is indoors, the position and orientation of the observation target 4 can be specified with higher accuracy. That is, in general, the geomagnetism tends to be disturbed indoors compared to the outdoors, and in particular, the geomagnetism tends to be disturbed as it approaches an electric product or concrete wall that generates electromagnetic waves. By actively using the disturbance, it is possible to specify the position of the observation target indoors more accurately.

  Further, according to the position specifying device 100A, the position and orientation closest to the candidate position and orientation based on the magnetic field matching result 140 are selected from the candidate positions and orientations based on the image matching result 120A, and the selected position and orientation are selected. Since the position and orientation of the observation target 4 in the positioning target space 5 are determined based on the azimuth, for example, when there are a plurality of reference data 20 including an image similar to an image photographed by the mobile terminal 3A attached to the observation target Even so, it becomes easy to specify the optimum position and orientation.

  In addition, as described above, the position specifying device 100A has the magnetic field vector Bj at the position 50_j that is a candidate by the magnetic field matching, the reference vector B0 included in the magnetic field map 22, and the reference of the magnetic field measured in advance in a specific direction. Since the azimuth candidates are determined based on the vector B0, the direction of the observation target can be specified more accurately.

<< Extended embodiment >>
Although the invention made by the present inventors has been specifically described based on the embodiments, it is needless to say that the present invention is not limited thereto and can be variously modified without departing from the gist thereof. Yes.

  For example, in the above-described embodiment, the case where the positioning target space is indoor has been described as an example, but it may be outdoor as long as the range is limited. In this case, as position information included in the reference data, information on latitude and longitude acquired by GPS may be used instead of coordinate information.

  In the above embodiment, the position specifying system 500, 500A includes a personal computer as the position specifying device 100, 100A including the data processing control unit 1, 1A and the storage unit 2, 2A, and the portable terminal 3, 3A. Although the structure connected by the cable etc. was illustrated, it is not restricted to this.

For example, as in the so-called cloud service, the position specifying system 500, 500A may be configured such that the server as the position specifying apparatus 100, 100A and the mobile terminal 3 are connected to a network formed by a wireless communication line. In the position specifying systems 500 and 500A, the observation target data 321 and 321A photographed by the portable terminals 3 and 3A are sequentially transferred to the server at a predetermined timing, and the server relates to the above-described position specification based on the observation target data 321. Processing may be executed. In this case, another monitoring terminal (personal computer) may be connected to the network, and the server may appropriately transmit the result of the processing relating to the position specification to the monitoring terminal.
According to this, it is possible to monitor the movement of the worker carrying the portable terminals 3 and 3A by the monitoring terminal installed at a place different from the worker.

  Moreover, although the case where the information of the azimuth | direction on the XY plane in the positioning object space 5 was memorize | stored as the azimuth | direction information 202 memorize | stored in reference | standard data was demonstrated in the said embodiment, it is not restricted to this. For example, the azimuth information 202 may include information on the component in the Z-axis direction, that is, information on the angle with respect to the XY plane.

  DESCRIPTION OF SYMBOLS 100,100A ... Position identification apparatus, 1, 1A ... Data processing control part, 2, 2A ... Memory | storage part, 3, 3A ... Portable terminal, 4 ... Observation object, 5 ... Positioning object space, 11, 11A ... Data acquisition part, DESCRIPTION OF SYMBOLS 12, 12A ... Image collation part, 13, 13A ... Identification part, 14 ... Magnetic field collation part, 20 ... Reference | standard data, 22 ... Magnetic field map, 31 ... Imaging device, 32 ... Memory | storage part, 33 ... Output part, 34 ... Magnetic sensor , 50_1 to 50_n, 50_i, 50_j ... positions in the positioning target space, 120,120A ... image matching results, 140 ... magnetic field matching results, 201 ... position information, 202 ... azimuth information, 203,322 ... images, 321,321A ... Observation target data, 323... Magnetic field information, 500.

Claims (5)

Stores reference data associating position information indicating a specific position in the positioning target space, direction information indicating an azimuth at the specific position, and an image shot at the specific position facing a predetermined direction. A storage unit;
Based on observation target data including an image photographed by an imaging device included in the observation target and the reference data stored in the storage unit, the position of the observation target and the direction of the observation target in the positioning target space are determined. A data processing control unit to identify,
The data processing control unit
A data acquisition unit for acquiring the observation target data;
An image collation unit that collates an image included in the observation target data acquired by the data acquisition unit with an image included in the reference data;
The position and orientation of the observation target are specified based on the position information and the azimuth information of the reference data including the image determined to match or similar to the image included in the observation target data by the verification by the image verification unit A position specifying device, comprising: The position identifying device according to claim 1,
The storage unit further stores a magnetic field map including information on a magnetic field in the positioning target space,
The observation target data further includes magnetic field information at a position where an image is captured by the imaging device,
The data processing control unit
A magnetic field collation unit for collating the magnetic field map with information on the magnetic field included in the observation target data;
The specifying unit specifies a position and an orientation of the observation target in the positioning target space based on a matching result by the image matching unit and a matching result by the magnetic field matching unit. The position specifying device according to claim 2,
The observation target data includes a first vector indicating the magnitude and direction of a magnetic field measured when an image is captured by the imaging device,
The magnetic field map includes a second vector indicating the magnitude and direction of the magnetic field measured at the specific position in the positioning target space, the magnitude of the magnetic field measured in a predetermined direction in the positioning target space, and A third vector indicating the direction,
The magnetic field collation unit collates the first vector and the second vector, determines a position candidate based on the collation result, the second vector at the candidate position, the first vector, A position specifying device that determines azimuth candidates based on the third vector. The position specifying device according to claim 3,
The image matching unit determines position and orientation candidates based on the reference data of an image that matches or is similar to an image included in the observation target data,
The specifying unit selects a position and orientation closest to the position and orientation candidate determined by the magnetic field verification unit from the position and orientation candidates determined by the image verification unit, and selects the selected position and orientation. The position specifying device characterized in that the position and orientation of the observation target in the positioning target space. The position specifying device according to any one of claims 1 to 4,
A portable terminal having the imaging device,
The position specifying system, wherein the portable terminal and the position specifying device are connected by wire or wireless.

Images