JP7396336B2 - Location information acquisition device, location information acquisition method, and program - Google Patents

Description

The present invention relates to a location information acquisition device, a location information acquisition method, and a program.

A technique has been disclosed in which a plurality of cameras image a marker and specify its three-dimensional position (for example, see Patent Document 1).

International Publication No. 2005/124687

However, the technology in the above patent document is susceptible to the effects of the optical characteristics of the camera and small positional deviations of the marker, and there is a concern that this may cause variations in the reliability of identifying three-dimensional positions. No solution had been thought of.

The present invention has been made in view of these problems, and aims to help solve the problem of variations in the positions of objects from which position information is to be obtained.

In order to achieve the above object, a location information acquisition device according to the present invention includes:
image acquisition means for acquiring images of the object captured by each of a first camera, a second camera, and a third camera installed at different locations;
The object as a pair of cameras used to derive the three-dimensional position of the object among the three cameras based on the position or size of the object in the image acquired by the image acquisition means. determining means for determining a pair that satisfies a predetermined reliability, which is an index set such that the closer the distance to the object is, the higher the reliability is;
Deriving means for deriving a three-dimensional position of the object based on imaging by the pair of cameras determined by the determining means;
It is characterized by having the following.

In order to achieve the above object, the location information acquisition method according to the present invention includes:
an image acquisition step of acquiring images of the object captured by each of a first camera, a second camera, and a third camera installed at different locations;
The object as a pair of cameras used to derive the three-dimensional position of the object among the three cameras based on the position or size of the object in the image acquired in the image acquisition step. a determining step of determining a pair that satisfies a predetermined reliability, which is an index set such that the closer the distance between the camera is to the object, the higher the reliability;
a deriving step of deriving a three-dimensional position of the object based on imaging by the pair of cameras determined in the determining step;
It is characterized by including.

In order to achieve the above object, the program according to the present invention:
computer,
image acquisition means for acquiring images of the object captured by each of a first camera, a second camera, and a third camera installed at different locations;
The object as a pair of cameras used to derive the three-dimensional position of the object among the three cameras based on the position or size of the object in the image acquired by the image acquisition means. a determining means for determining a pair that satisfies a predetermined reliability, which is an index set such that the closer the distance to the object is, the higher the reliability is;
deriving means for deriving a three-dimensional position of the object based on imaging by the pair of cameras determined by the determining means;
It is characterized by functioning as

According to the present invention, it is possible to help solve the problem of variations in the positions of objects from which position information is to be obtained.

1 is a diagram showing an example of a visible light communication system according to an embodiment of the present invention. It is a diagram showing an example of the configuration of a server according to the same embodiment. FIG. 3 is a diagram illustrating an example of parallax determined from images captured by two cameras according to the same embodiment. FIG. 6 is a diagram illustrating an example of calculation of installation positions and imaging directions of two cameras according to the same embodiment. It is a figure which shows an example of speed calculation based on the same embodiment. It is a figure showing an example of space division concerning the same embodiment. 12 is a flowchart illustrating an example of reliability information acquisition by the server according to the embodiment. It is a flowchart which shows an example of the acquisition process of the installation position of a 2nd light source by the server based on the same embodiment. 12 is a flowchart illustrating an example of generation and retention of a location/reliability information table by a server according to another embodiment. FIG. 7 is a diagram showing an example of a position/reliability information table according to another embodiment. 12 is a flowchart illustrating another example of generation and retention of a location/reliability information table by a server according to another embodiment. FIG. 7 is a diagram showing an example of a position/reliability information table according to another embodiment. FIG. 7 is a diagram illustrating an example of position error calculation according to another embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS A visible light communication system as a position information acquisition system according to an embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram showing the configuration of a visible light communication system. As shown in FIG. 1, the visible light communication system 1 includes devices 100a, 100b, and 100c installed in a space 500 (hereinafter referred to as "devices 100" when each of the devices 100a, 100b, and 100c is not limited to each other). ) and a server 200 corresponding to the location information acquisition device.

The device 100a has a second light source 102a attached to it, the device 100b has a second light source 102b attached to it, and the device 100c has a second light source 102c attached to it (hereinafter referred to as second light sources 102a, 102b, and 102c, respectively). (If not limited, it will be appropriately referred to as the "second light source 102"). The server 200 is equipped with cameras 201a, 201b, 201c, and 201d that correspond to imaging devices (hereinafter, the cameras 201a, 201b, 201c, and 201d will be appropriately referred to as "cameras 201" unless they are limited to each other). . In addition, first light sources 300a, 300b, 300c, 300d, and 300e are installed in the space 500 (hereinafter, when each of the first light sources 300a, 300b, 300c, 300d, and 300e is not limited, appropriate (referred to as "first light source 300"). The first light source 300 and the second light source 102 include not-shown LEDs (Light Emitting Diodes). The second light source 102 corresponds to a position information acquisition target.

In this embodiment, the second light source 102 attached to the device 100 transmits information by emitting light corresponding to various types of information to be transmitted, such as the status of the device 100. On the other hand, the server 200 demodulates the change in the luminescent color in the light images obtained by continuous time-series imaging by the camera 201 and acquires the information emitted by the second light source 102 .

In this embodiment, the positions and imaging directions of the cameras 201a to 201d are initially unknown. Therefore, before the server 200 acquires the status of the device 100, etc., the server 200 first checks the first light sources 300a, 300b, 300c, 300d, and 300e in the images obtained by the cameras 201a to 201d. Based on the position (two-dimensional coordinate information) of each image, the positions (installation positions) and imaging directions of the cameras 201a to 201d in the space 500, which is a three-dimensional space, are calculated. Furthermore, the server 200 generates a conversion matrix for converting the position (two-dimensional coordinate information) of each image of the first light source 300 in the image obtained by imaging into a position (installation position) in the space 500.

FIG. 2 is a diagram showing an example of the configuration of the server 200. As shown in FIG. 2, the server 200 includes a control section 202, an image input section 204, a memory 205, an operation section 206, a display section 207, and a communication section 208. Furthermore, cameras 201a to 201d are attached to the server 200 via wiring.

The camera 201a includes a lens 203a, the camera 201b includes a lens 203b, the camera 201c includes a lens 203c, and the camera 201d includes a lens 203d. (If not, it will be appropriately referred to as "lens 203"). The lens 203 is composed of a zoom lens or the like. The lens 203 is moved by a zoom control operation from the operation unit 206 and focusing control by the control unit 202. By moving the lens 203, the angle of view and optical image captured by the camera 201 are controlled.

The cameras 201a to 201d are composed of a plurality of light receiving elements regularly arranged two-dimensionally on a light receiving surface. The light receiving element is, for example, an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The cameras 201a to 201d capture (receive) optical images incident through the lens 203 at a predetermined range of imaging angles of view based on control signals from the control unit 202, and receive image signals within the imaging angles of view. converts into digital data and generates a frame. Furthermore, the cameras 201a to 201d temporally continuously capture images and generate frames, and output consecutive frames to the image input unit 204 in the server 200.

A frame (digital data) output from the camera 201 is input to the image input unit 204 based on a control signal from the control unit 202 .

The control unit 202 is configured by, for example, a CPU (Central Processing Unit). The control unit 202 controls various functions provided by the server 200 by executing software processing according to a program stored in the memory 205 (for example, a program for realizing the operation of the server 200 shown in FIG. 3, which will be described later). do.

The memory 205 is, for example, RAM (Random Access Memory) or ROM (Read Only Memory). The memory 205 stores various information (programs, etc.) used for control etc. in the server 200.

The operation unit 206 is an interface that includes a numeric keypad, function keys, etc., and is used to input the contents of a user's operation. The display unit 207 is configured by, for example, an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), an EL (Electro Luminescence) display, or the like. The display unit 207 displays an image according to the image signal output from the control unit 202. The communication unit 208 is, for example, a LAN (Local Area Network) card. The communication unit 208 communicates with an external communication device under the control of the communication control unit 242.

The control unit 202 includes an image processing unit 231, a camera position/imaging direction calculation unit 232, a matrix generation unit 234, a light emitting position acquisition unit 236 corresponding to the calculation means, and an imaging situation acquisition unit corresponding to the information acquisition means. 238, a reliability information acquisition section 240 corresponding to reliability information acquisition means, and a communication control section 242.

The image processing unit 231 performs peripheral dimming correction and distortion correction on the frames (digital data) output from the camera 201 and input to the image input unit 204 in order to display them as through images on the display unit 207, and improves the image quality. or adjust the image size. Further, when a control signal based on a recording instruction operation from the operation unit 206 is input, the image processing unit 231 controls the image processing unit 231 to display an image within the imaging angle of view of the camera 201 at the time when the recording instruction is issued or a display displayed on the display unit 207. It has a function of encoding an optical image within the range using a compression encoding method such as JPEG (Joint Photographic Experts Group) and converting it into a file. The camera position/imaging direction calculation unit 232 detects the position (two-dimensional coordinate information) of the image of the first light source 300 in each image obtained by imaging by the cameras 201a to 201d. Here, the installation positions (three-dimensional coordinate information) of the first light sources 300a, 300b, 300c, 300d, and 300e in the space 500 are known. In addition, the first light sources 300a, 300b, 300c, 300d, and 300e have a three-color pattern of R (red), G (green), and B (blue) that is modulated by an ID (Identification) that can uniquely identify itself. It emits light that changes cyclically.

The camera position/imaging direction calculation unit 232 sets a combination of two cameras 201 (camera pair) for the cameras 201a to 201d. There are six patterns (six ways) of combinations (camera pairs) of any two cameras 201 out of the four cameras 201.

The camera position/imaging direction calculation unit 232 detects a cyclic tricolor pattern of light included in each image obtained by imaging by the cameras 201a to 201d. Further, the camera position/imaging direction calculation unit 232 attempts to detect the ID corresponding to the pattern of the three-color light emission and demodulate the ID. The memory 205 stores the installation positions and IDs of each of the first light sources 300a, 300b, 300c, 300d, and 300e in association with each other.

Furthermore, the camera position/imaging direction calculation unit 232 calculates, for each camera pair, modulated light that is light modulated by the same ID from both images obtained by imaging by the two cameras 201 included in the camera pair. An attempt is made to detect a light area (a pixel area of a specific size and shape that has a high luminance value that is higher than a preset value). If the first light source 300 corresponding to the ID can be detected, it is assumed that the first light source 300 corresponding to the ID has been detected. Further, the camera position/imaging direction calculation unit 232 recognizes the number of detected first light sources 300 for each camera pair.

Next, for each camera pair, the camera position/imaging direction calculation unit 232 calculates the number of first light sources 300 included in both images captured by the two cameras 201 included in the camera pair. An algorithm for calculating the positions (installation positions) and imaging directions of the two cameras 201 included in the pair in the space 500 is set. For example, the algorithm is a 5-point algorithm when the number of first light sources 300 included in each image captured by the camera pair is 5, and an 8-point algorithm when there are 8 first light sources 300. They are prepared for each light source 300 and stored in the memory 205.

Next, the camera position/imaging direction calculation unit 232 calculates the installation positions and imaging directions of the two cameras 201 included in the camera pair for each camera pair using the set algorithm.

The algorithm will be explained below. FIG. 3 is a diagram illustrating an example of parallax determined from an image captured by the camera 201. Further, FIG. 4 is a diagram showing an example of calculation of the installation position and imaging direction of the camera 201.

As shown in FIG. 3, when two cameras 201 (here, cameras 201a and 201b) included in a camera pair capture images of the same first light source 300c, since the installation positions of the cameras 201a and 201b are different, The position (two-dimensional coordinate information) of the image 251a of the first light source 300c in the image of the imaging surface 250a obtained by imaging with the camera 201a and the image of the first light source 300c in the image of the imaging surface 250b obtained by imaging with the camera 201b. A difference (parallax) S occurs between the position (two-dimensional coordinate information) of 251b.

Further, as shown in FIG. 4, the distance from the imaging surface 250a of one camera 201 (here, camera 201a) of the two cameras 201 (here, cameras 201a and 201b) included in the camera pair to the focal position , the distance from the imaging surface 250b of the other camera 201 (camera 201b here) to the focal position is F (same value), the distance between the installation positions of camera 201a and camera 201b is B, and the focal position of camera 201a is The shortest distance between the first light source 300c and the straight line connecting the camera 201b and the focal position of the camera 201b is D, and the distance between the first light source 300c and the first light source 300c obtained by virtually overlapping the imaging plane 250a and the imaging plane 250b is If the parallax between the position of the image 251a and the position of the image 251b is S, then the distance calculation formula D=B×F/S holds true. In this equation, F and S are known fixed values.

In this embodiment, the camera position/imaging direction calculation unit 232 calculates a distance calculation formula not only for the first light source 300c but also for each of the detected first light sources 300. Furthermore, the camera position/imaging direction calculation unit 232 calculates the installation positions and positions of the two cameras 201 included in the camera pair based on the obtained distance calculation formula and the installation position of the first light source 300 that has been measured in advance. Calculate the imaging direction.

Specifically, the camera position/imaging direction calculation unit 232 calculates the position of the image 251a of the first light source 300 (Xga1 , Yga1) and the position (Xgb1, Ygb1) of the image 251b of the first light source 300 included in the image captured by the other camera 201, the relative installation positions and imaging directions of the two cameras 201 are determined. seek.

Next, the camera position/imaging direction calculation unit 232 refers to the IDs of the first light sources 300a to 300e, reads out the installation positions of the first light sources 300a to 300e stored in the memory 205, and reads out the installation positions of the first light sources 300a to 300e. are used to calculate the installation positions and imaging directions of the two cameras 201 included in the camera pair in the space 500. Then, based on the calculated installation positions and imaging directions of one camera 201 and the other camera 201, the matrix generation unit 234 generates an image of the first light source 300 included in the image obtained by imaging with one camera 201. The installation of the first light source 300 in the space 500 is based on the combination of the position (two-dimensional coordinate information) and the position (two-dimensional coordinate information) of the image of the first light source 300 included in the image obtained by imaging with the other camera 201. A transformation matrix that can be transformed into a position (position information defined by three-dimensional spatial coordinates) is determined. A transformation matrix is determined for each camera pair.

The second light sources 102a, 102b, and 102c emit light that changes cyclically in a three-color pattern of R (red), G (green), and B (blue), which is modulated with an ID that can uniquely identify itself.

After the transformation matrix is determined for each camera pair, the light emitting position acquisition unit 236 detects the cyclic three-color pattern of light included in each image obtained by the cameras 201a to 201d. Further, the light emitting position acquisition unit 236 attempts to detect an ID corresponding to this three-color light emission pattern and demodulate the ID. If the light emission position acquisition unit 236 is able to detect the same ID from both images obtained by the two cameras 201 included in the camera pair, the light emission position acquisition unit 236 detects the second light source 102 corresponding to the ID. is considered to be able to be detected.

Next, the light emission position acquisition unit 236 determines, for each camera pair, the position (Xga2, Yga2) of the image of the second light source 102 on the imaging surface of one of the two cameras 201 included in the camera pair. and the position (Xgb2, Ygb2) of the image of the second light source 102 on the imaging surface of the other camera 201. Further, the light emission position acquisition unit 236 uses the combination of the positions (Xga2, Yga2) and (Xgb2, Ygb2) of both images and the transformation matrix to determine the installation position (Xk2, Yk2, Zk2) are calculated.

Through the above-described processing, the installation position of one second light source 102 may be acquired in correspondence with each of a plurality of camera pairs. In such a case, reliability information (likelihood information) of the installation position of the second light source 102 in the space 500 is acquired for each camera pair. The acquisition of reliability information will be explained below.

The imaging situation acquisition unit 238 determines, for each camera pair, that the position of the image of the second light source 102 in the image acquired from the imaging surface of one of the two cameras 201 included in the camera pair is determined in the image. The closer the image is to the center, the higher the reliability regarding the position of the image is set, and the farther the distance from the center, the lower the reliability regarding the position of the image is set. Similarly, the imaging situation acquisition unit 238 sets the reliability regarding the position of the second light source 102 in the image obtained from the imaging surface of the other camera 201 to be higher as the position of the image of the second light source 102 is closer to the center of the image. However, the farther the distance from the center is, the lower the reliability of the image position is set. This setting process is related to the distortion correction process in the image processing unit 231, and in this distortion correction process, the closer to the center of the imaging plane the weaker the correction strength of the distortion correction is, and the closer to the periphery the lower the correction strength of the distortion correction. Make it stronger. Therefore, the closer the position of the image of the second light source 102 is to the periphery, the more the positional shift occurs due to distortion correction, and as a result, the reliability becomes lower.

Through the above-described processing, reliability information (image position reliability information) B1 regarding the image position with respect to the one second light source 102 of interest is obtained for one camera 201 of the two cameras 201 included in the camera pair. For the other camera 201, reliability information (image position reliability information) B2 regarding the image position with respect to the one second light source 102 of interest is acquired.

Further, the imaging situation acquisition unit 238 refers to a plurality of frames (images) acquired by successive imaging by one camera 201 of the two cameras 201 included in the camera pair, and The moving speed of the second light source 102 is calculated from the change in the position of the light source 102.

FIG. 5 is a diagram showing an example of speed calculation. For example, the position of the image of the second light source 102 in frame F1 among a plurality of frames F1, F2 to Fn continuously captured in the time direction t is 710a, and the position of the image of the second light source 102 in frame Fn is 710a. is 710n.
Then, consider a case where the distance between image positions 710a and 710n in frame Fx, which is a superimposition of frames F1 and Fn, is L. In this case, the imaging situation acquisition unit 238 determines the distance L and , the moving speed of the second light source 102 can be calculated based on the frame rate. Further, the imaging situation acquisition unit 238 sets the reliability regarding the moving speed so that the slower the moving speed (the smaller the distance L), the higher the speed reliability.

Similarly, the imaging situation acquisition unit 238 controls the second light source 102 in each of the images acquired by continuous imaging by the camera 201 of the other camera 201 of the two cameras 201 included in the camera pair. The moving speed of the second light source 102 is calculated from the distance L of the image position. Further, the imaging situation acquisition unit 238 sets the reliability regarding the moving speed so that the slower the moving speed (the smaller the distance L), the higher the speed reliability.

The above-mentioned processing is performed even if the second light source 102 is fixed at the same installation position for a long period of time, the imaging angle of the camera 201 changes minutely, and/or the second light source 102 moves minutely. This is done because the installation location may change depending on the situation. Specifically, reliability information (speed reliability information) C1 regarding the moving speed with respect to the second light source 102 is acquired for one camera 201 of the two cameras 201 included in the camera pair, and , reliability information (speed reliability information) C2 regarding the moving speed with respect to the same second light source 102 is acquired.

Furthermore, the imaging situation acquisition unit 238 divides the space 500 into a plurality of regions. FIG. 6 is an example of space division. In FIG. 6, the image is divided into three parts in the vertical direction at equal intervals, and also divided into three parts in the horizontal direction at equal intervals, resulting in nine regions (divided regions) 501a, 501b, 501c, 501d, 501e, and 501f. , 501g, 501h, and 501i (hereinafter referred to as "divided areas 501" as appropriate unless each of the divided areas 501a to 501i is limited) are formed.

Further, the imaging situation acquisition unit 238 determines the second light source 102 based on the relative positional relationship between the installation positions of the two cameras 201 included in the camera pair and the installation position of the one second light source 102 of interest. Reliability information (installation position reliability information) D regarding the installation position of the light source 102 is acquired.

Specifically, the imaging situation acquisition unit 238 sets the reliability of the installation position to be high because the closer the installation position of the two cameras 201 included in the camera pair is, the larger the image of the second light source 102 can be captured. . For example, when the camera 201a and the camera 201d are installed as shown in FIG. 6, if the second light source 102 exists in the divided areas 501a, 501b, and 501c, the installation position reliability is high; If it exists in the divided areas 501e and 50f, the installation position reliability is medium, and if it exists in the divided areas 501g, 501h, and 501i, the installation position reliability is low.

In addition, for each camera pair, the reliability information acquisition unit 240 determines, for the image of the first light source 300 captured by both of the two cameras 201 included in the camera pair, the reliability information acquisition unit 240 Using the combination of the position (two-dimensional coordinate information), the position (two-dimensional coordinate information) obtained by imaging by the other camera 201, and the transformation matrix corresponding to the camera pair, the first The installation position of the light source 300 (position information defined by three-dimensional spatial coordinates) is calculated. Furthermore, the reliability information acquisition unit 240 calculates the error between the calculated installation position of the first light source 300 in the space 500 and the installation position (known information) of the first light source 300 stored in the memory 205. Furthermore, the reliability information acquisition unit 240 sets the error reliability A so that the smaller the error, the higher the reliability regarding the error (error reliability).

Next, the reliability information acquisition unit 240 acquires, for each camera pair, image position reliability information B1, B2, speed reliability information C1, C2, installation position reliability information D, error reliability information acquired by the above-described processing. Using information A, reliability information N regarding installation position calculation for one second light source 102 of interest (reliability information of second light source 102) is calculated. For example, it is calculated by N=A×(B1+B2+C1+C2+D).

Thereafter, the light emitting position acquisition unit 236 calculates the installation position of the second light source 102 within the space 500. At this time, the installation position of one second light source 102 of interest may be calculated from each of a plurality of camera pairs. In such a case, the light emission position acquisition unit 236 compares the reliability information N acquired for each camera pair with respect to the one second light source 102 of interest. Then, the light emission position acquisition unit 236 selects the camera pair corresponding to the highest reliability information N.

Next, the light emission position acquisition unit 236 determines the position (two-dimensional coordinate information) of the image of the second light source 102 in the image obtained by imaging by one camera 201 of the selected camera pair, and The position (two-dimensional coordinate information) of the image of the second light source 102 in the image obtained by imaging is acquired. Furthermore, the light emission position acquisition unit 236 uses the combination of the positions of these two images and the transformation matrix to determine the installation position of the second light source 102 in the space 500 (position information defined by three-dimensional spatial coordinates). calculate.

Next, the operation of the server 200 will be explained with reference to a flowchart. FIG. 7 is a flowchart illustrating an example of reliability information acquisition by the server 200. The operation shown in FIG. 7 is performed for each camera pair and for each first light source 300 imaged by both of the two cameras included in the camera pair.

The two cameras 201 included in one camera pair image the same first light source 300, and identify the first light source 300 by acquiring the ID (step S101).

Next, the imaging status acquisition unit 238 determines whether the position of the image of the second light source 102 in the image captured by one of the two cameras 201 included in the camera pair is the same as that of the captured image. The image position reliability information B1 is acquired so that the closer to the center the higher the image position reliability, and the position of the image of the second light source 102 in the image taken by the other camera 201 is determined. The image position reliability information B2 is acquired such that the closer to the center of the image the higher the image position reliability is (step S102).

Next, the imaging situation acquisition unit 238 determines the moving speed of the second light source 102 based on each of the images acquired by continuous imaging by one of the two cameras 201 included in the camera pair. is calculated, and the speed reliability information C1 is acquired such that the slower the moving speed, the higher the speed reliability. Similarly, the imaging situation acquisition unit 238 calculates the moving speed of the second light source 102 based on each of the images acquired by continuous imaging by the other camera 201, and the slower the moving speed, the higher the speed reliability. The speed reliability information C1 is acquired such that the speed reliability information C1 becomes high (step S103).

Next, the imaging situation acquisition unit 238 determines whether the installation position of the second light source 102 is included in the camera pair based on the positional relationship between the installation positions of the two cameras 201 included in the camera pair and the installation position of the second light source 102. The installation position reliability information D is acquired such that the closer to the installation positions of the two included cameras 201, the higher the installation position reliability is (step S104).

Next, for the image of the first light source 300 captured by both of the two cameras 201 included in the camera pair, the reliability information acquisition unit 240 determines the position obtained by capturing the image of one camera 201 and the position of the other camera 201. The installation position of the first light source 300 in the space 500 is calculated using a combination of positions obtained by imaging by the camera 201 and a transformation matrix corresponding to the camera pair. Furthermore, the reliability information acquisition unit 240 calculates the error between the calculated installation position of the first light source 300 in the space 500 and the installation position (known information) of the first light source 300 stored in the memory 205. , the error reliability information A is acquired such that the smaller the error, the higher the error reliability (step S105).

Furthermore, the reliability information acquisition unit 240 uses the acquired image position reliability information B1, B2, speed reliability information C1, C2, installation position reliability information D, and error reliability information A to determine the accuracy of the second light source 102. Reliability information N is acquired (step S106).

FIG. 8 is a flowchart illustrating an example of a process of acquiring the installation position of the second light source 102 by the server 200. The plurality of cameras 201 capture images of the second light source 102 in the space 500 (step S201).

Next, for one second light source 102, if there are a plurality of camera pairs in which both of the two cameras 201 are imaging the second light source 102, the light emission position acquisition unit 236 The highest reliability information is selected from among the reliability information of the second light source 102 that has been acquired. Further, the light emitting position acquisition unit 236 selects a camera pair corresponding to the selected reliability information (step S202).

Next, the light emitting position acquisition unit 236 determines the position of the image of the second light source 102 in the image obtained by imaging by one camera 201 of the selected camera pair, and the position of the image of the second light source 102 in the image obtained by imaging by the other camera 201 of the selected camera pair. and the position of the image of the second light source 102 in the image. Furthermore, the light emitting position acquisition unit 236 calculates the installation position of the second light source 102 using the combination of these two acquired positions and the transformation matrix (step S203).

Next, the light emitting position acquisition unit 236 determines whether installation positions have been calculated for all the second light sources 102 imaged in step S201 (step S204). If the installation positions of all the second light sources 102 have been calculated (step S204; YES), the series of operations ends. Furthermore, if there is a second light source 102 whose installation position has not been calculated (step S204; NO), the operations from step S202 onwards are repeated.

In this way, in this embodiment, the server 200 determines, for each camera pair, the position of the image of the second light source 102 in the image acquired by both the two cameras 201 included in the camera pair. Reliability information regarding the calculation of the installation positions of the two light sources 102 is acquired. Further, the server 200 calculates the installation position of the second light source 102 based on the image acquired by the camera pair. At this time, if the installation position of the second light source 102 can be calculated for each camera pair by capturing images of the second light source 102 by a plurality of camera pairs, the server 200 calculates the reliability of the second light source 102. The camera pair with the highest value is selected, and the installation position of the second light source 102 is calculated based on the image captured by that camera pair. Thereby, it becomes possible to calculate the installation position of the second light source 102 based on the imaging of the camera pair with high reliability of the second light source 102, and it is possible to improve the accuracy of calculation.

Specifically, the server 200 acquires the image position reliability information such that the closer the image position of the second light source 102 is to the center of the captured image, the higher the image position reliability. As a result, the reliability of the second light source 102 can be lowered as the image position of the second light source 102 is farther from the center of the image, and the distortion rate becomes larger as the distance from the center increases. This makes it possible to obtain reliable reliability information.

The server 200 also calculates the speed of the second light source 102 based on the captured image, and acquires the speed reliability information such that the slower the speed, the higher the speed reliability. This makes it possible to obtain appropriate reliability information in accordance with the fact that the faster the moving speed is, the lower the calculation accuracy of the installation position of the second light source 102 is.

Further, the server 200 acquires the installation position reliability information such that the closer the installation position of the second light source 102 is to the installation positions of the two cameras 201 included in the camera pair, the higher the installation position reliability is. . Thereby, in general triangulation, it becomes possible to obtain appropriate reliability information in accordance with the fact that the farther the distance from the camera 201 is, the lower the calculation accuracy of the installation position is.

Further, the server 200 calculates an error between the calculated installation position of the first light source 300 in the space 500 and the known information between the installation position of the first light source 300, and the smaller the error, the higher the error reliability becomes. The error reliability information is obtained as follows. This makes it possible to give priority to a camera pair with a small error, that is, with high calculation accuracy, and use it for calculating the installation position of the second light source 102.

Next, other embodiments will be described. In this embodiment, the visible light communication system 1 is the same as that shown in FIG. 1, and the server 200 is the same as that shown in FIG. 2. In this embodiment, a plurality of installation positions of the first light source 300 and the second light source 102 are calculated for one light source, and reliability information is set for each installation position.

FIG. 9 is a flowchart illustrating an example of generation and retention of a location/reliability information table by the server 200 according to another embodiment. The operation shown in FIG. 9 is performed for each first light source 300.

For each camera pair, when the two cameras 201 included in the camera pair image the same first light source 300, the captured image is acquired via the image input unit 204, and the control unit 202 acquires the control unit ID. An attempt is made to identify the first light source 300 (step S301).

Next, the camera position/imaging direction calculation unit 232 selects a camera pair whose ID was able to be captured by capturing an image of the first light source 300 in step S301 (step 302).

Next, the light emission position acquisition unit 236 calculates the installation position of the first light source 300 for each camera pair selected in step S302, based on images captured by the two cameras 201 included in the camera pair (step S303). Specifically, similar to step S203 in FIG. 8, the light emission position acquisition unit 236 determines the position of the image of the first light source 300 in the image obtained by one camera 201 of the camera pair and The position of the image of the first light source 300 in the image obtained by the camera 201 is acquired. Further, the light emission position acquisition unit 236 calculates the installation position of the first light source 300 using the combination of these two acquired positions and the transformation matrix corresponding to the camera pair.

Next, the reliability information acquisition unit 240 generates a position/reliability information table 2051 for the first light source 300 whose installation position was calculated in step 303, and stores it in the memory 205 (step S304).

FIG. 10 is a diagram showing an example of the position/reliability information table 2051 generated and held in a predetermined storage area of the memory 205 in step S304. The position/reliability information table 2051 shown in FIG. 10 includes, for each first light source 300 that is a light source, the ID of the first light source 300 and the installation position obtained by imaging by the camera pair that captured the first light source 300. , information on the camera pair that captured the image used to calculate the installation position, reliability information on the installation position, update date and time indicating the date and time when the installation position was calculated, and an error.

The reliability information is set in three stages, A, B, and C, in descending order of reliability. Regarding the first light source 300, the reliability information acquisition unit 240 acquires image position reliability information acquired in the same manner as step S102 in FIG. 7, speed reliability information acquired in the same manner as in step S103 in FIG. The reliability information is set by appropriately selecting the installation position reliability information obtained in the same manner as step S104 in FIG. 7, the error reliability information obtained in the same manner as in step S105 in FIG.

The error is set in three stages: R1, R2, and R3 in descending order of error. For example, the reliability information acquisition unit 240 sets the error so that the closer the update date and time is to the present, the smaller the error becomes.

FIG. 11 is a flowchart illustrating another example of generation and retention of a location/reliability information table by the server 200 according to another embodiment. The operation shown in FIG. 11 is performed for each second light source 102.

For each camera pair, the two cameras 201 included in the camera pair capture an image of the same second light source 102 and attempt to identify the second light source 102 by acquiring an ID (step S401).

Next, the camera position/imaging direction calculation unit 232 selects a camera pair whose ID was able to be captured by capturing an image of the second light source 102 in step S401 (step 402).

Next, the light emitting position acquisition unit 236 calculates the installation position of the second light source 102 for each camera pair selected in step S402, based on images captured by the two cameras 201 included in the camera pair (step S402). S403). Specifically, similar to step S203 in FIG. 8, the light emission position acquisition unit 236 determines the position of the image of the second light source 102 in the image obtained by one camera 201 of the camera pair and the position of the image of the second light source 102 in the image obtained by one camera 201 of the camera pair. The position of the image of the second light source 102 in the image obtained by the camera 201 is acquired. Furthermore, the light emission position acquisition unit 236 calculates the installation position of the second light source 102 using the combination of these two image positions and the transformation matrix corresponding to the camera pair.

Next, the reliability information acquisition unit 240 generates position/reliability information for the second light source 102 whose installation position was calculated in step 403, and additionally stores it in the position/reliability information table 2051 (step S404).

FIG. 12 is a diagram showing an example of the location/reliability information table 2052 in step S404. FIG. 12 shows the position/reliability information table 2051 generated for each first light source 300 shown in FIG. 10 in which the position/reliability information generated for each second light source 102 is added in step S404. A degree information table 2052 is shown.

Similarly to the position/reliability information table 2051 generated for each first light source 300, the position/reliability information table 2052 generated for each second light source 102 contains the ID of the second light source 102 and the The installation position obtained by imaging by the camera pair that captured the light source 102, information on the camera pair that captured the image used to calculate the installation position, reliability information on the installation position, and the date and time when the installation position was calculated. It is composed of the update date and time indicating the update date and the error.

The reliability information is set in three stages, A, B, and C, in descending order of reliability. Regarding the second light source 102, the reliability information acquisition unit 240 acquires image position reliability information acquired in the same manner as step S102 in FIG. 7, speed reliability information acquired in the same manner as in step S103 in FIG. The reliability information is set by appropriately selecting the installation position reliability information obtained in the same manner as step S104 in FIG. 7, the error reliability information obtained in the same manner as in step S105 in FIG.

Further, the reliability information acquisition unit 240 may acquire the installation position of the first light source 300 for the second light source 102, and acquire the installation position, reliability information, and error of the second light source 102. good.

The error is set in three stages: R1, R2, and R3 in descending order of error. For example, the reliability information acquisition unit 240 sets the error so that the closer the update date and time is to the present, the smaller the error becomes.

Next, the light emitting position acquisition unit 236 determines whether installation positions have been calculated for all the second light sources 102 imaged in step S401 (step S405). If the installation positions of all the second light sources 102 have been calculated (step S405; YES), the series of operations ends. Furthermore, if there is a second light source 102 whose installation position has not been calculated (step S405; NO), the operations from step S402 onwards are repeated.

In this way, by generating and maintaining the position/reliability information table for the first light source 300 and the second light source 102, the calculated reliability information of the plurality of installation positions of the first light source 300 and the second light source 102 is calculated. is obtained. Therefore, for the first light source 300 and the second light source 102, it is possible to select the installation position with the highest reliability, or to select the most appropriate installation position that takes into account both reliability information and error. .

Furthermore, for the first light source 300 and the second light source 102 for which only installation positions with low reliability are calculated, it is assumed that the installation positions cannot be specified, or appropriate installation positions are calculated taking into account both reliability information and error. For the first light source 300 and second light source 102 that are not installed, it is possible to appropriately specify the installation positions, such as by considering that the installation positions cannot be specified.

Note that even if only the position/reliability information table 2051 is generated, it is possible to subsequently obtain the installation position, reliability information, and error of the second light source 102. As a specific example, a case will be described in which after only the position/reliability information table 2051 is generated and the first light source 300 is removed from the space 500, the cameras 201a to 201d image the inside of the space 500. As shown in FIG. 13, for the space 500, after the position/reliability information table 2051 is generated and maintained, the first light source 300 is removed and the second light source 102c is newly installed ( normal operating condition). Then, when the cameras 201a to 201d capture images of the space 500 in this state and the captured images are respectively input to the image input unit 204, the image processing unit 231 detects the second light source 102c from these images. Furthermore, the installation position of the second light source 102c is calculated from the image and determinant of the second light source 102c in these captured images. Then, referring to the position/reliability information table 2051, the first light source 300 imaged by the camera pair closest to the calculated installation position of the second light source 102c and having high reliability (in FIG. 300d and 300e). Note that the error E of the second light source 102c is determined by the following method. As shown in FIG. 13, the distance in the X direction between the first light source 300d and the second light source 102c in space is Xf, the distance in the X direction between the first light source 300e and the second light source 102c is Xg, and the first light source If the error between the position of the first light source 300d based on known information and the calculated position is Ef, and the error between the position of the first light source 300e based on the known information and the calculated position is Eg, then the error E of the position of the second light source 102c is , is calculated by interpolation as E=(Ef*Xg+Eg+Xf)/((Xf+Xg).

Note that the present invention is not limited by the description of the embodiment and the drawings, and it is possible to make changes to the embodiment and the drawings as appropriate.

For example, in the embodiment described above, the reliability information acquisition unit 240 stores, for each camera pair, image position reliability information B1, B2, speed reliability information C1, C2, and installation position reliability information acquired by the processing described above. D. Using error reliability information A, reliability information N for one second light source 102 of interest was calculated by N=A×(B1+B2+C1+C2+D).

However, the calculation formula is not limited to this, and for example, image position reliability information B1, B2, speed reliability information C1, C2, installation position reliability information D, and error reliability information A may all be multiplied. Further, the reliability information acquisition unit 240 appropriately selects the image position reliability information B1, B2, the speed reliability information C1, C2, the installation position reliability information D, and the error reliability information A to obtain information about the second light source 102. The reliability information N may be calculated. For example, if the highest reliability information of the second light source 102 is less than a threshold value, the error reliability information A may not be multiplied. Furthermore, for example, the installation position reliability information D may be calculated for each of the two cameras 201 included in the camera pair.

Further, in the embodiment described above, the server 200 selects the camera pair with the highest reliability of the second light source 102, and calculates the installation position of the second light source 102 based on the image captured by the camera pair. However, the method of calculating the installation position is not limited to this.

For example, the server 200 calculates the installation position of the second light source 102 based on the imaging of the camera pair for all camera pairs in which both of the two cameras 201 are imaging the second light source 102, and calculates the installation position of the second light source 102 based on the imaging of the camera pair. The higher the reliability of the light source 102 in a camera pair, the greater the weighting of the installation position of the second light source 102 calculated based on the imaging of the camera pair. Further, the server 200 may calculate an average value of the installation positions of the second light source 102 calculated based on images taken by a plurality of camera pairs having the highest reliability of the second light source 102.

Furthermore, in the embodiment described above, as shown in FIG. 6, if the space 500 is divided into nine divided regions 501a to 501i and the second light source 102 is present in the divided regions 501a, 501b, and 501c, the installation position is reliable. The reliability information is high, if it exists in the divided areas 501d, 501e, and 50f, the installation position reliability information is medium, and if it exists in the divided areas 501g, 501h, and 501i, the installation position reliability information is low. The settings of the divided regions 501 and the installation reliability information corresponding to each divided region 501 are not limited to this. Installation reliability information may be different for each divided area 501.

In addition, regarding the first light source 300, the installation position in the space 500 was stored in the memory 205 in association with each ID, but by visible light communication, light modulated according to the installation position in the space 500 may be made to emit light.

Further, in the embodiment described above, the reliability information in the location/reliability information table is set in three stages A, B, and C in descending order of reliability, but is not limited to this and can be set in more stages. It may be set as , or it may be set as a numerical value. In addition, the reliability information is set by appropriately selecting image position reliability information, speed reliability information, installation position reliability information, error reliability information, etc., but is not limited thereto.

Furthermore, the errors in the position/reliability information table are set in three stages, R1, R2, and R3, in descending order of error; however, the error is not limited to this, and may be set in more stages. may be set. Further, although the error is set to be smaller as the update date and time is closer to the current time, the error is not limited to this.

For example, the first light source 300 and the second light source 102 are not limited to LEDs. For example, a light source may be included in a part of an LCD, PDP, EL display, etc. that constitute a display device.

Further, the server 200 may be any device as long as a camera can be attached thereto.

In the above embodiments, the program to be executed may be a computer-readable record such as a flexible disk, a CD-ROM (Compact Disc - Read Only Memory), a DVD (Digital Versatile Disc), or an MO (Magneto - Optical Disc). A system that executes the above processing may be configured by storing and distributing the program on a medium and installing the program.

Further, the program may be stored in a disk device or the like of a predetermined server on a network such as the Internet, and may be downloaded by being superimposed on a carrier wave, for example.

In addition, if the above functions are realized by the OS (Operating System) or by cooperation between the OS and applications, it is possible to store only the parts other than the OS on a medium and distribute it. You may also download it.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and the present invention includes the inventions described in the claims and their equivalents. It will be done. Below, the invention described in the original claims of the present application will be added.

(Additional note 1)
A location information acquisition system comprising a plurality of imaging devices and a location information acquisition device,
The location information acquisition device includes:
By detecting light based on the identification information from images of a predetermined space in which there are a plurality of location information acquisition targets that emit light based on the identification information, which are imaged by the plurality of imaging devices from different directions. , calculation means for calculating a three-dimensional position of the location information acquisition target in the predetermined space;
information acquisition means for acquiring information regarding the imaging status of the plurality of imaging devices;
Reliability information of the three-dimensional position of the position information acquisition target in the predetermined space calculated by the calculation means is acquired based on information regarding the imaging situation of the plurality of imaging devices acquired by the information acquisition means. Reliability information acquisition means;
A location information acquisition system comprising:

(Additional note 2)
The position information acquisition system according to appendix 1, wherein the information regarding the imaging status of the plurality of imaging devices is information regarding the optical characteristics of the imaging devices.

(Additional note 3)
3. The positional information acquisition system according to appendix 1 or 2, wherein the information regarding the imaging status of the plurality of imaging devices is information regarding the displacement of the positional information acquisition target.

(Additional note 4)
The imaging device sequentially captures images and outputs images,
The position information according to appendix 3, wherein the displacement of the position information acquisition target is determined from the presence or absence of movement of the position information acquisition target by comparing a plurality of images output by the imaging device. acquisition system.

(Appendix 5)
a holding unit that holds the reliability information acquired by the reliability information acquisition unit in association with identification information of the location information acquisition target corresponding to the reliability information;
Calculation control means for controlling the calculation means to calculate the positions of the plurality of position information acquisition targets from the images captured by each of the plurality of imaging devices after the reliability information acquisition means obtains the reliability information. and,
The position information acquisition system according to any one of Supplementary Notes 1 to 4, comprising:

(Appendix 6)
The number of the imaging devices is at least three or more,
The calculation means calculates the position of the plurality of positions by detecting light based on identification information included in an image captured by an imaging device constituted by a set of at least two of three or more imaging devices. Calculating a position defined in multiple dimensions in the space of information acquisition target,
The holding means stores the reliability information acquired by the reliability information acquisition means as information specifying a set consisting of identification information of the location information acquisition target corresponding to the reliability information and the at least two imaging devices. The position information acquisition system according to appendix 5, wherein the position information acquisition system is stored in association with each other.

(Appendix 7)
Detecting light based on the identification information from images captured by each of a plurality of imaging devices that capture images from different directions of a predetermined space in which a plurality of location information acquisition targets exist that emit light based on the identification information. a calculation means for calculating the three-dimensional position of the location information acquisition target in the predetermined space;
information acquisition means for acquiring information regarding the imaging status of the plurality of imaging devices;
Reliability information of the three-dimensional position of the position information acquisition target in the predetermined space calculated by the calculation means is acquired based on information regarding the imaging situation of the plurality of imaging devices acquired by the information acquisition means. Reliability information acquisition means;
A location information acquisition device comprising:

(Appendix 8)
The position information acquisition device according to appendix 7, wherein the information regarding the imaging status of the plurality of imaging devices is information regarding the optical characteristics of the imaging devices.

(Appendix 9)
9. The location information acquisition device according to appendix 7 or 8, wherein the information regarding the imaging status of the plurality of imaging devices is information regarding the displacement of the location information acquisition target.

(Appendix 10)
The imaging device sequentially captures images and outputs images,
The positional information according to appendix 9, wherein the displacement of the positional information acquisition target is determined from the presence or absence of movement of the positional information acquisition target by comparing a plurality of images output by the imaging device. Acquisition device.

(Appendix 11)
a holding unit that holds the reliability information acquired by the reliability information acquisition unit in association with identification information of the location information acquisition target corresponding to the reliability information;
Calculation control means for controlling the calculation means to calculate the positions of the plurality of position information acquisition targets from the images captured by each of the plurality of imaging devices after the reliability information acquisition means obtains the reliability information. and,
The position information acquisition device according to any one of Supplementary Notes 7 to 10, comprising:

(Appendix 12)
The number of the imaging devices is at least three or more,
The calculation means calculates the position of the plurality of positions by detecting light based on identification information included in an image captured by an imaging device constituted by a set of at least two of three or more imaging devices. Calculating a position defined in multiple dimensions in the space of information acquisition target,
The holding means stores the reliability information acquired by the reliability information acquisition means as information specifying a set consisting of identification information of the location information acquisition target corresponding to the reliability information and the at least two imaging devices. The position information acquisition device according to appendix 11, characterized in that the position information acquisition device is associated and held.

(Appendix 13)
Detecting light based on the identification information from images captured by each of a plurality of imaging devices that capture images from different directions of a predetermined space in which a plurality of location information acquisition targets exist that emit light based on the identification information. a calculation step of calculating a three-dimensional position of the location information acquisition target in the predetermined space;
an information acquisition step of acquiring information regarding the imaging status of the plurality of imaging devices;
Reliability information of the three-dimensional position of the position information acquisition target in the predetermined space calculated in the calculation step is acquired based on information regarding the imaging status of the plurality of imaging devices acquired in the information acquisition step. Reliability information acquisition step;
A location information acquisition method characterized by comprising:

(Appendix 14)
computer,
Detecting light based on the identification information from images captured by each of a plurality of imaging devices that capture images from different directions of a predetermined space in which a plurality of location information acquisition targets exist that emit light based on the identification information. Calculating means for calculating the three-dimensional position of the location information acquisition target in the predetermined space;
information acquisition means for acquiring information regarding the imaging status of the plurality of imaging devices;
Reliability information of the three-dimensional position of the position information acquisition target in the predetermined space calculated by the calculation means is acquired based on information regarding the imaging situation of the plurality of imaging devices acquired by the information acquisition means. Reliability information acquisition means,
A program characterized by functioning as

1... Visible light communication system, 100, 100a, 100b, 100c... Equipment, 102, 102a, 102b, 102c... Second light source, 200... Server, 201, 201a, 201b, 201c, 201d... Camera, 202... Control unit, 203, 203a, 203b, 203c, 203d...Lens, 204...Image input section, 205...Memory, 206...Operation section, 207...Display section, 208...Communication section, 231...Image processing section, 232...Camera position/imaging direction Calculation unit, 234... Matrix generation unit, 236... Light emission position acquisition unit, 238... Imaging status acquisition unit, 240... Reliability information acquisition unit, 242... Communication control unit, 250a, 250b... Imaging surface, 300, 300a, 300b, 300c, 300d, 300e...First light source, 500...Space, 501, 501a, 501b, 501c, 501d, 501e, 501f, 501g, 501h, 501i...Divided area, 710a, 710b, 710n... Image position

Claims (4)

image acquisition means for acquiring images of the object captured by each of a first camera, a second camera, and a third camera installed at different locations;
The object as a pair of cameras used to derive the three-dimensional position of the object among the three cameras based on the position or size of the object in the image acquired by the image acquisition means. determining means for determining a pair that satisfies a predetermined reliability, which is an index set such that the closer the distance to the object is, the higher the reliability is;
Deriving means for deriving a three-dimensional position of the object based on imaging by the pair of cameras determined by the determining means;
A location information acquisition device comprising: The image acquisition means further acquires an image of a marker different from the target object imaged by each of the first camera, the second camera, and the third camera,
The determining means stores in a memory a three-dimensional position of the marker derived for each pair of images formed by combining two of the three images of the marker acquired by the image acquisition means. and the three-dimensional position of the marker, and select a pair of cameras that satisfies the predetermined reliability as a pair of cameras to be used for deriving the three-dimensional position of the object among the three cameras. The position information acquisition device according to claim 1, wherein the position information acquisition device determines the position information. an image acquisition step of acquiring images of the object captured by each of a first camera, a second camera, and a third camera installed at different locations;
The object as a pair of cameras used to derive the three-dimensional position of the object among the three cameras based on the position or size of the object in the image acquired in the image acquisition step. a determining step of determining a pair that satisfies a predetermined reliability, which is an index set such that the closer the distance between the camera is to the object, the higher the reliability;
a deriving step of deriving a three-dimensional position of the object based on imaging by the pair of cameras determined in the determining step;
A location information acquisition method characterized by comprising: computer,
image acquisition means for acquiring images of the object captured by each of a first camera, a second camera, and a third camera installed at different locations;
The object as a pair of cameras used to derive the three-dimensional position of the object among the three cameras based on the position or size of the object in the image acquired by the image acquisition means. a determining means for determining a pair that satisfies a predetermined reliability, which is an index set such that the closer the distance to the object is, the higher the reliability is;
deriving means for deriving a three-dimensional position of the object based on imaging by the pair of cameras determined by the determining means;
A program characterized by functioning as

Images