JP2006284442A - Method of monitoring object and motion tracker using this - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for monitoring the motion (position, angle) of an object which is attached with a marker having no discrimination information. <P>SOLUTION: A pair of cameras 12 and 14 for stereo-viewing an object are placed. Marker groups 18 with 4 or more turnable markers are attached and coordinates of each marker are defined on an object fixed coordinates system. Three markers with a position relation not forming a regular triangle when connecting the three markers are turned on as markers for searching. Based on geometrical prediction, correlation of marker images for searching making pairs between images is formed. For the three markers for searching, absolute coordinates are calculated and distance between the markers for searching is calculated. The markers are discriminated for identifying which of marker groups 18 the three markers for searching belong. Based on the absolute coordinates of the markers for searching and the coordinates on the object fixed coordinates system, the position and the angle of the object are identified. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

The present invention attaches a light emitter as a marker to an object having a certain shape, projects the marker with a pair of cameras capable of stereo viewing, and moves the object, that is, changes from moment to moment from the image of the marker projected on the camera. The present invention relates to an object monitoring method for detecting position information and angle information (direction) of an object to be detected, and a motion tracker using this method.
The present invention is used for, for example, a head motion tracker (HMT) that detects the position and angle of a helmet-type head-mounted display device used in a game machine, a vehicle, or the like.

  A technique for accurately monitoring the movement of an object having a fixed shape is used in various fields. For example, in order to realize virtual reality (VR) in a game machine or the like, a moving image is displayed using a head-mounted display device. At this time, the movement of the head is monitored, and the movement is displayed. It is necessary to change the display image at the same time, and a head motion tracker (HMT) for detecting the movement of the head-mounted display device which is an object having a fixed shape is attached.

A head motion tracker (HMT) attached to a head-mounted display device uses a magnetic source to generate a magnetic field in the surrounding space, and measures magnetism with a magnetic sensor fixed to the head-mounted display device. A magnetic measurement system is disclosed that detects the current position and current angle of a magnetic sensor from the measured magnetic data, and consequently the current position and current angle of the head-mounted device (see Patent Document 1).
Although the magnetic measurement method can be used without any limitation as long as a magnetic field is generated in the space in which the magnetic sensor moves, it is necessary to measure magnetic information in the space in which the magnetic sensor moves in advance and perform magnetic mapping. is there. In addition, if an object that causes magnetostriction such as metal exists in this space, a measurement error occurs due to the influence, and it is necessary to correct it.

On the other hand, what optically measures the position and angle of a head-mounted display device is also disclosed. For example, by attaching a reflector to a head-mounted display device, irradiating light from a light source, photographing the reflected light with a camera, and measuring the position and angle of the marker using the video data Have been disclosed (see Patent Document 2).
Since the optical measurement method does not require magnetic mapping and is not affected by metal or the like, the optical method is easy to use in an environment or place that is susceptible to magnetostriction.
JP 2002-81904 A JP-T 9-506194

If an optical motion tracker is used to accurately measure 3D position information and angle information (orientation) for an object of a fixed shape, it is necessary to know the positions of three different points that are fixed on the object. is there. Therefore, the position is measured by attaching markers each identifiable to three different places on the object.
For example, a helmet-shaped head-mounted display device will be specifically described. On the outer peripheral surface of the helmet of the head-mounted display device, LEDs that emit infrared light having different wavelengths are used as markers to identify individual markers. As described above, the positions of the three markers are fixed at the three positions, and the positional relationship of these three markers on the coordinate system (referred to as the helmet coordinate system) fixed to the helmet is measured and determined in advance. By shooting these three points simultaneously with each camera using two cameras that can be viewed in stereo and whose installation location is fixed (ie, the distance between the two cameras is known), After identifying the LED (marker) from the image of the LED (marker), the absolute coordinates of the three markers (coordinates determined with respect to the ground space) are measured by the so-called triangulation principle. .
Then, for these three marker positions, the absolute coordinate and the positional relationship of the helmet coordinate system fixed on the head-mounted display device that has been measured in advance are associated with each other to thereby match the head-mounted display device with respect to the absolute coordinate. Specify the position and angle.

  In the optical system as described above, since it is necessary to identify each marker, the marker itself has some information that can be distinguished from other markers so that it can be identified on the detection side (camera side). Yes. In the case of the above example, the marker is identified by changing the emission wavelength for each marker and identifying the emission wavelength on the camera side.

However, when each marker has identifiable information, it is necessary to store the identification information again every time the marker is replaced due to a failure or the like.
Also, if the position and direction of the head-mounted display device changes significantly compared to the camera field of view, one of the three markers may be out of the camera field of view due to the relationship between the marker position and the camera installation position. There is no stereo viewing.
For this reason, if three markers that can be identified are always displayed on two cameras at the same time so that each marker can be viewed in stereo, the markers that can be identified are distributed over the entire circumference of the helmet. In addition, the positional relationship of all these markers on the helmet coordinate system is measured in advance, so that at least three positions within the field of view of the two cameras can be obtained regardless of the position and angle of the helmet. There must be an identifiable marker. In this case, it is necessary to give some identification information (for example, different emission wavelengths) so that all attached markers can be identified, and management of identification information and marker arrangement is difficult.

  Also, if you want to identify individual markers with markers that do not have identification information, you can identify them by lighting them one by one, but in that case, all the markers must be lit sequentially, Since it takes a long time to complete a cycle, it is difficult to monitor the movement that changes from moment to moment.

  Therefore, the present invention optically measures the positions of three different markers displayed on the camera from among a large number of markers attached to the object, and monitors the movement (position and angle) of the object from these three movements. In this case, the marker itself is not provided with information for distinguishing it from other markers, but the individual marker is identified only from the marker position information, so that the marker position, and hence the object to which the marker is attached. An object of the present invention is to provide a method for monitoring the movement (position, angle).

  In addition, the present invention does not monitor each individual marker by lighting it one by one, but lights up three markers necessary for measurement simultaneously and measures the positions of these markers almost simultaneously. Thus, an object is to provide a monitoring method capable of monitoring even when movement is intense.

  In addition, the present invention uses a marker that does not have identification information, and does not measure while sequentially identifying each marker separately, and lighting three markers simultaneously. An object of the present invention is to provide a motion tracker that monitors the movement of an object by measuring these markers almost simultaneously.

The object monitoring method of the present invention made in order to solve the above-mentioned problems is as follows: (a) a pair of cameras for stereo-viewing an object; and (b) at least four illuminable marker groups on the outer surface of the object. Mounting, (c) determining the coordinates of each marker on the object fixed coordinate system defined on the object,
(D) Of the marker group, three markers that are simultaneously reflected in the images of the pair of cameras, and the three markers that connect the three markers are in a positional relationship that does not form an equilateral triangle, are used as search markers. Illuminates, and (e) the three search marker images shown in each image are associated with each other based on the geometric prediction, and (f) the association With respect to the three search marker images that have been made, the absolute coordinates of each search marker on the absolute coordinate system defined in space are calculated by triangulation using a pair of cameras, and (g) the calculated 3 The distance between the search markers is calculated from the absolute coordinates of each of the two search markers, and (h) the distance between the markers obtained from the coordinates of each marker on the previously determined object fixed coordinate system is calculated. Based on the relationship between the distance between the three search markers, the marker identification that identifies which of the three search markers is a marker group attached to the object, respectively ( i) The position and angle of the object with respect to the absolute coordinate system are specified based on the absolute coordinates of each of the three search markers specified by the marker identification and the coordinates on the object fixed coordinate system.

According to the monitoring method of the present invention, among the markers attached to the outer surface of the object, three markers that are included in the field of view of the pair of cameras and can be simultaneously reflected in the images of the respective cameras are used as search markers. Light.
When the search marker image is displayed on each image of the pair of cameras, the geometric information is obtained from the position information of each camera and the shooting direction of each camera, and the positional relationship between the search marker images displayed on the two images. By the prediction (for example, prediction by epipolar geometry), a pair of search markers that are paired between two images can be associated.
At this stage, only a set of search markers between the respective images is specified, and it cannot be identified which of the three search markers.
When a pair of search markers that are paired between two images is specified, the direction angle from each camera for each search marker can be specified, and so-called triangulation can be performed from the distance between the cameras and these direction angles. By doing so, the absolute coordinates of each search marker can be calculated.
When the absolute coordinates of each of the three search markers are calculated, the distance between the search markers can be calculated. This is compared with the distance between the markers determined from the coordinates of the markers on the previously defined object fixed coordinate system. Since the triangle connecting the three search markers does not form an equilateral triangle, the length of at least one of the sides is different from the other two sides, based on the difference in distance between the three search markers. It is possible to specify which of the marker groups attached to the object each search marker is.
When the three search markers are specified, the position of the object with respect to the absolute coordinate system is determined based on the absolute coordinates calculated for each of the three search markers and the coordinates on the predetermined object fixed coordinate system. The angle can be calculated.
In this way, the movement of the object is monitored by obtaining the position and angle of the object with respect to the absolute coordinate system.

  According to the present invention, it is possible to optically identify a marker from the positional relationship of three markers displayed simultaneously on a pair of cameras without having identification information for each marker attached to the object. And angle can be monitored.

In addition, according to the present invention, three markers necessary for measurement are lit at the same time without illuminating each marker and monitoring while identifying each marker, and the positions of these markers are almost the same. Since it can be measured simultaneously, the position and angle can be monitored even when the movement is intense.
(Means and effects for solving other problems)

  Following the steps (a) to (i) of the invention of the object monitoring method, (j) based on the position and angle of the object with respect to the absolute coordinate system and the coordinates of each marker on the object fixed coordinate system Calculating the absolute coordinates of all the markers, (k) for all the markers for which the absolute coordinates have been calculated, calculating the planned positions where each marker appears on each image of the pair of cameras, and (l) all the markers or searching. The remaining markers except for the marker are turned on, and the markers are identified for all the markers by associating the planned positions on each image with the actual marker image positions. (M) Among the identified markers The three markers that appear simultaneously in each image of a pair of cameras are extracted as new search markers, and the absolute coordinates of these three new search markers are obtained by triangulation using a pair of cameras. Calculated, it may be identified again by the position and angle of the object relative to the absolute coordinate system based on the coordinates on (n) 3 single new search marker each absolute coordinates and the object fixed coordinate system.

According to this invention, the absolute coordinates of all the markers can be calculated based on the position and angle of the object with respect to the absolute coordinate system and the coordinates of each marker on the predetermined object fixed coordinate system. For all markers for which absolute coordinates have been calculated (or for the remaining markers except for the search marker that has already been identified), the estimated positions where each marker appears on each image of a pair of cameras are calculated and all The marker is turned on, and marker identification for each marker is performed by associating the calculated estimated position on each image with the marker image position actually reflected. As a result, all markers appearing on the image can be identified. After that, three identified markers appearing simultaneously in each image of a pair of cameras are extracted as new search markers, and these are determined by so-called triangulation from the relationship between the camera position and the new search marker image position shown in the image. The absolute coordinates of the three new search markers are calculated, and the position and angle of the object with respect to the absolute coordinate system are specified again based on the three new search markers.
As a result, even when the object changes from moment to moment and the marker displayed on the image changes, the position and angle of the object are calculated based on the three new search markers simultaneously displayed on the images of the pair of cameras at that time. be able to.

In (a) in the above invention, the pair of cameras may be installed so that a part of the object is viewed in stereo.
According to this, since the distance between the camera and the object can be reduced, monitoring can be performed without taking a place. Further, by reducing the distance between the camera and the object, the position of the marker attached to the object can be accurately measured, and the accuracy of monitoring can be increased.

Further, in (b) in the above invention, the position on the object to which the marker group is attached is such that when a part of the object is projected by a pair of cameras, at least three markers of these marker groups are always at the same time a pair of cameras. The marker groups may be distributed over the entire outer surface of the object so as to be reflected in each of the above.
According to this, regardless of the direction of the object, three markers can be projected on a pair of cameras, and the position and angle of the object can be calculated using these as search markers. It can be monitored without any restrictions on the direction.

Moreover, in (l) in the said invention, you may make it perform matching with a plan position and a marker image position based on the distance between both.
When the distance between the planned position and the marker position is equal to or less than a certain value, the marker can be identified with high accuracy by specifying the marker.

  Further, the motion tracker of the present invention made from another viewpoint includes a pair of cameras for performing a stereo view of an object, at least four illuminable marker groups attached to the outer surface of the object, and a pair of cameras and marker groups. The motion tracker includes a control calculation unit that performs control and calculation for obtaining the position and angle of the object, and the marker group defines the coordinates of each marker on the object fixed coordinate system defined on the object. The control calculation unit searches for three markers in the marker group, which are three markers that appear in the images of the pair of cameras at the same time, and in which the triangle connecting these three markers does not form an equilateral triangle. Search markers that illuminate as markers and are paired between images based on geometric predictions for the three search marker images shown in each image Associating each other and calculating the absolute coordinates of each search marker on the absolute coordinate system defined on the space by triangulation using a pair of cameras for the three associated search marker images Then, the distance between the search markers is calculated from the absolute coordinates of each of the calculated three search markers, and the distance between the markers obtained from the coordinates of each marker on the previously determined object fixed coordinate system is calculated. Based on the relationship between the three search marker distances, each of the three search markers is identified from among a group of markers attached on the object to identify the marker. Based on the absolute coordinates of each of the search markers and the coordinates on the object fixed coordinate system, an operation for specifying the position and angle of the object with respect to the absolute coordinate system is performed. .

  In the motion tracker, the control calculation unit further calculates the absolute coordinates of all the markers based on the position and angle of the identified object with respect to the absolute coordinate system and the coordinates of each marker on the object fixed coordinate system. Calculate and calculate the estimated positions where each marker appears on each image of a pair of cameras for all markers for which absolute coordinates have been calculated, and turn on the remaining markers excluding all markers or search markers The markers are identified for all the markers by associating the planned positions on each of the images and the marker image positions actually reflected, and among the identified markers, three markers that are simultaneously displayed on each image of the pair of cameras are displayed. These three new search markers are extracted as new search markers and based on the relationship between the camera position and the new search marker image position displayed in the image. -By calculating the absolute coordinates of the car and re-identifying the position and angle of the object with respect to the absolute coordinate system based on the absolute coordinates of each of the three new search markers and the coordinates on the object fixed coordinate system. You may make it monitor the position and angle which change.

  Hereinafter, an object monitoring method according to an embodiment of the present invention will be described using a head motion tracker using this method as a specific example with reference to the drawings. However, the present invention is not limited to the specific examples described below, and can be applied without departing from the spirit of the present invention.

(Embodiment 1)
FIG. 1 is a diagram showing a schematic configuration of a head motion tracker according to an embodiment of the present invention.
The head motion tracker 10 includes a pair of cameras 12 and 14 installed so as to be arranged at a distance d so as to enable stereo viewing, a helmet 16, and at least four or more LEDs attached to the outer surface of the helmet 16. It comprises a group 18 (marker group) and a control calculation unit 20 constituted by a computer.

The cameras 12 and 14 are each directed to the helmet 16 in the shooting direction, and the LED group 18 so that at least three LEDs (the same LED in the cameras 12 and 14) of the LED group 18 enter the field of view at the same time. The installation position of the camera is determined according to the number and arrangement of the cameras.
The cameras 12 and 14 may be installed at positions slightly apart so that the entire image of the helmet 16 (the entire image facing the camera) falls within the field of view. In this case, even if the number of LEDs attached to the helmet 16 is relatively small, three LEDs can be included in the field of view of the cameras 12 and 14.

  On the other hand, the cameras 12 and 14 may be brought close to the helmet so that only a part of the helmet falls within the field of view. In this case, the number of LED groups 18 attached to the helmet 16 must be increased so that at least three LEDs are included in the field of view, but the position of the LEDs can be measured with high accuracy.

  Moreover, you may adjust the range which attaches LED group 18 on the helmet 16 according to the range which the helmet 16 moves. That is, when the range of movement of the helmet 16 is small, the range of fluctuation of the field of view of the cameras 12 and 14 is also small, so the LED group 18 may be attached only in that region. When the helmet 16 moves in all directions, it is necessary to attach the LED group 18 over the entire helmet 16. It should be noted that the number of LEDs attached to the helmet 16 as the LED group 18 works more effectively as the number increases, but the cost also increases, so it is necessary to make it an appropriate number.

The LED group 18 is not limited in emission wavelength as long as the lighting state can be confirmed by the cameras 12 and 14. For example, when an infrared camera is used as the camera, an infrared light emitting LED can be used.
The position of each LED in the LED group 18 on the helmet, that is, the coordinates of each LED in the coordinate system (helmet coordinate system) fixed on the helmet, is measured, and thus the distance between the LEDs is known.

  The control calculation unit 20 performs various calculation processes in conjunction with the controls on the cameras 12 and 14 and the LED group 18. The control and calculation contents executed by the control calculation unit 20 will be described separately for each function. As shown in FIG. 1, the control calculation unit 20 includes a search marker lighting unit 21, a search marker association unit 22, and a search use. A marker absolute coordinate calculation unit 23, a search marker distance calculation unit 24, a search marker identification unit 25, and an object position / angle identification unit 26 are provided.

The search marker lighting section 21 is an LED group 18 (marker group) of three LEDs (markers) that are simultaneously shown in the images of the pair of cameras 12 and 14, and the triangle connecting these LEDs is a regular triangle. The three LEDs that are in a positional relationship that does not form are lit as search LEDs (search markers). Other markers must be off.
The search marker associating unit 22 performs pairing between images by prediction based on epipolar geometry for three search LED images (search marker images) reflected in each of two images by the pair of cameras 12 and 14. The search LED images are matched.
The search marker absolute coordinate calculation unit 23 calculates each search LED absolute coordinate from the relationship between the camera position and the search LED image position displayed in the image for the three search LED images associated with each other. .
The search marker distance calculation unit 24 calculates the distance between the search LEDs from the absolute coordinates of the calculated three search LEDs.
The search marker identifying unit 25 is based on the relationship between the distance between the LEDs obtained from the coordinates of the LEDs on the helmet coordinate system (object fixed coordinate system) and the calculated distance between the three search LEDs. Marker identification is performed to identify which of the three search LEDs is one of the LED groups 18 mounted on the object.
The object position / angle specifying unit 26 determines the helmet 16 (with respect to the absolute coordinate system) based on the absolute coordinates of the three search LEDs specified by the marker identification and the coordinates on the helmet coordinate system (object fixed coordinate system). The position and angle of the (object) are specified.

Next, the object monitoring by the head motion tracker 10, that is, the operation of monitoring the position and angle every moment will be described.
FIG. 2 is a diagram showing an operation flow when the position and angle of the measurement target object are monitored by the head motion tracker 10.

  First, as shown in FIG. 3, three LEDs in the field of view of the two cameras 12 and 14 (however, the three positional relationships are not equilateral triangles) are lit as search markers, and the other LED groups 18 Is turned off (s101). For convenience, it is assumed that these three LEDs are a set of LEDs 18a, 18b, and 18c.

It is confirmed that three LEDs that are lit are reflected in the images A and B of the two cameras 12 and 14 (s102). FIG. 4 shows an image example at this time. Among the many LEDs, only three LEDs are lit. In order to ensure that the three LEDs fall within the field of view of the cameras 12 and 14, the initial position of the helmet 16 may be designated and initially the helmet wearer may be prompted to set it near the initial position. If any of the three LEDs is not shown in the field of view, the set of the three LEDs to be lit may be changed and shown.
At this stage, it is known that each of the three LED images shown in the images A and B is one of the LEDs 18a, 18b, and 18c, but each of the LED images is any one of the three. It is not specified.

Therefore, association for recognizing a common LED set (search marker set) between the images A and B is performed by prediction based on epipolar geometry (s103). FIG. 5 is a diagram for explaining prediction based on epipolar geometry.
With the shooting direction of the camera 12 and the shooting direction of the camera 14 determined, the image A of the camera 12 has an LED 18x that is one of the LEDs 18a, 18b, and 18c that are lit (LED 18y and LED 18z are not shown). Is projected. At this time, assuming that there is a virtual straight line L connecting the camera 12 and the LED 18x, and this is projected by the camera 14, the virtual straight line LB is projected on the image B of the camera 14. The position of the virtual straight line LB on the image B is uniquely determined based on the position where the LED 18x appears in the image A and the relationship between the shooting direction of the camera 12 and the shooting direction of the camera 14. The image of the LED 18x exists at any position on the virtual straight line LB. Therefore, if there is an LED image that is displayed on the image B and is on the virtual straight line LB, it can be specified that it is an image of the LED 18x. Similarly, by specifying the image of the other two LEDs 18y and the image of the LED 18z, it is possible to associate a set of three LEDs 18x, LED 18y, and LED 18z displayed in the image A and the image B.

Subsequently, the absolute coordinates of the three search markers LED18x, LED18y, and LED18z are calculated, and the three-dimensional positions of these three LEDs with respect to the absolute space are specified (s104). FIG. 6 is a diagram for explaining calculation of absolute coordinates by triangulation.
The absolute coordinates of the LED 18x can be uniquely calculated if the direction angle αx from the camera 12 to the LED 18x, the direction angle βx from the camera 14 to the LED 18x, and the inter-camera distance d are known. The direction angle αx can be determined by the shooting direction of the camera 12 and the position of the LED 18x in the image A. Similarly, the direction angle βx can be determined by the shooting direction of the camera 14 and the position of the LED 18x in the image B. it can. Since the inter-camera distance d is known, the absolute coordinates of the LED 18x are calculated from αx, βx, d. Similarly, absolute coordinates are calculated for the LEDs 18y and 18z.

  Subsequently, distances between the three search markers LED 18x, LED 18y, and LED 18z are calculated (s105). Since the absolute coordinates of each of these three LEDs are calculated, the distance dx between the LED 18x and the LED 18y, the distance dy between the LED 18y and the LED 18z, and the distance dz between the LED 18z and the LED 18x can be obtained by simple calculation.

  Subsequently, the search markers for specifying which of the three search markers LED 18x, LED 18y, and LED 18z are the LEDs 18a, 18b, and 18c, respectively, are identified (s106). The search marker is identified by comparing the distances dx, dy, dz between the LEDs 18x, 18y, 18d and the distances a, b, c between the LEDs 18a, 18b, 18c, which are previously known in the helmet coordinate system. The If the distances a, b, and c between the LEDs 18a, 18b, and 18c are all equal (that is, the positional relationship of the LEDs 18a, 18b, and 18c is an equilateral triangle), the search marker cannot be identified by the distance, but it is lit 3 Since one LED that does not have a regular triangular positional relationship is selected, identification is possible.

Subsequently, when the three LEDs are identified and the LEDs 18a, 18b, and 18c are specified, the helmet relative to the absolute coordinates is obtained by associating the positions in the absolute coordinate system and the positions in the helmet coordinate system with respect to these three points. 16 positions and angles (orientations) are calculated (s107).
With the above calculation processing, the position and angle of the helmet 16 with respect to the space (absolute coordinates) can be determined, and the position and orientation of the helmet can be specified.

(Embodiment 2)
FIG. 7 is a diagram showing a schematic configuration of a head motion tracker 30 according to another embodiment of the present invention. In FIG. 7, the same components as those in FIG.
In the second embodiment, functions are further added to the first embodiment, and the control calculation unit 40 includes an all marker absolute coordinate calculation unit 41, an all marker planned position calculation unit 42, and all A marker identification unit 43, a new marker absolute coordinate calculation unit for search 44, and an object position / angle re-specification unit 45 are added. In the second embodiment, all LEDs (all markers) attached to the helmet 16 are identified, new search markers are extracted from these, and the position and angle of the helmet that changes from moment to moment are changed by the new search markers. It is for monitoring.

The absolute marker calculation unit 41 includes a position and an angle relative to the absolute coordinate system of the helmet 16 (object) specified by the object position angle specifying unit 26, and each LED (each marker on the helmet coordinate system (object fixed coordinate system)). ) To calculate the absolute coordinates of all the LEDs.
The all marker planned position calculation unit 42 calculates the planned positions at which the respective LEDs on the images A and B of the two cameras 12 and 14 appear for all LEDs (all markers) for which absolute coordinates have been calculated.
The all marker identifying unit 43 turns on all the LEDs and performs marker identification for all the LEDs by associating the planned positions on the two images A and B with the marker image positions actually reflected in the images A and B. Do.
The new search marker absolute coordinate calculation unit 44 extracts three LEDs that are simultaneously displayed in the two images A and B of the cameras 12 and 14 among the identified LEDs as new search LEDs (new search markers), From the relationship between the camera position and the new search LED image position displayed in the images A and B, the absolute coordinates of these three new search LEDs are calculated.
The object position / angle respecifying unit 45 re-identifies the position and angle of the object with respect to the absolute coordinate system based on the absolute coordinates of the three new search LEDs and the coordinates on the helmet coordinate system. Monitor the changing position and angle.

  FIG. 8 is a diagram illustrating an operation flow when the position and angle of the helmet 16 are monitored by the head motion tracker 30. In this operation flow, the procedure from s201 to s207 is the same as the procedure from s101 to s107 in FIG. 2 in the first embodiment, so that the description thereof will be omitted and the subsequent procedure will be described.

  As described in the first embodiment, in s207, three LEDs that are search LEDs are identified to identify LEDs 18a, 18b, and 18c, and these three identified search LEDs (LEDs 18a, 18b, and 18c) are identified. Based on this, it is assumed that the position and angle (orientation) of the helmet 16 with respect to the absolute coordinates are calculated (s207).

  Subsequently, the absolute coordinates of all LEDs are calculated based on the position and angle of the helmet 16 with respect to the absolute coordinate system and the coordinates of each LED on the helmet coordinate system (s208).

  Subsequently, for all LEDs for which absolute coordinates have been calculated, the expected positions where the respective LEDs are expected to appear on the images A and B of the cameras 12 and 14 are determined as the shooting directions of the cameras 12 and 14 and the absolute coordinates of the LEDs. (S209).

Subsequently, all the LEDs are turned on (s210), and the estimated positions of the LEDs on the images A and B are associated with the positions of the LED images actually reflected in the images A and B. Marker identification that identifies which of the two is 18 is performed (s211). As shown in FIG. 9, it is determined whether or not one LED image exists within a predetermined distance from the planned position of each LED, as shown in FIG. When only one LED image exists, all the LEDs are associated. Thereby, the marker identification which is which of the LED group 18 is performed about all the LEDs. The search LEDs (that is, the LEDs 18a, 18b, and 18c) are the second marker identification. However, the markers are again identified for accuracy confirmation.
At the time of marker identification, if at least one LED image deviates from a predetermined distance from the planned position (for example, when the helmet changes rapidly and greatly), the association is not performed and is ignored.

  Subsequently, among all the LEDs, three LEDs that appear simultaneously in the two images A and B of the cameras 12 and 14 are extracted as new search LEDs (new search markers), and are displayed in the camera position and the images A and B. From the relationship with the new search LED image position, the absolute coordinates of these three new search LEDs are calculated (s212). The three LEDs that will be the new search LEDs may or may not be the same as the search LEDs selected during s201 (same as s101), depending on changes in the position and orientation of the helmet 16. .

  Subsequently, for the three new search LEDs, the position and angle of the helmet are determined by re-identifying the position and angle of the helmet 16 with respect to the absolute coordinate system based on the calculated absolute coordinates and the coordinates on the helmet coordinate system. Are monitored (s213).

Subsequently, three new search LEDs (new search markers) are turned on (s214). At this time, the other LEDs are turned off.
Subsequently, it is confirmed whether images of three new search LEDs (new search markers) are displayed on the images of the two cameras (s215). If the images are not projected on the two cameras at the same time, the process proceeds to s210 by using the scheduled position obtained immediately before in s209, and the subsequent operations are repeated.
On the other hand, when the images are simultaneously projected on the two cameras, the process proceeds to s203 (the new search marker is replaced with the search marker), and the subsequent operations are repeated.


With the above operation, it is possible to track the change in the position and angle of the helmet, which changes every moment, that is, the movement of the helmet.

  In the above operation flow, all the LEDs are turned on in s210, but instead of this, the three previously lit LEDs as the search LEDs may be turned off and the remaining LEDs may be turned on. In this case, marker identification is performed for the remaining LEDs, but marker identification has already been performed for the search LED, and therefore marker identification of all LEDs is possible. Further, in this case, since the number of lighting in s210 can be reduced, the possibility of erroneous LED identification can be reduced as much as possible, and more reliable processing can be performed.

  In the above embodiment, the head motion tracker attached to the helmet is taken as an example. However, the present invention is not limited to this, and the head motion tracker can be used as a motion tracker for other objects having a certain shape.

  The present invention can be used for an optical motion tracker that uses a light emitter having no identification information.

The figure which shows the structure of the head motion tracker which is one Embodiment of this invention. The flowchart explaining the operation | movement by the head motion tracker of FIG. The figure explaining the use condition in the head motion tracker of FIG. The figure which shows the example of an image projected on a camera. The figure explaining the prediction based on epipolar geometry. The figure explaining calculation of the absolute coordinate by triangulation. The figure which shows the structure of the head motion tracker which is other one Embodiment of this invention. 8 is a flowchart for explaining an operation by the head motion tracker of FIG. The figure explaining the marker identification of each LED performed based on the positional relationship between the calculated estimated position and each LED.

Explanation of symbols

10 Head motion tracker 12, 14 Camera 16 Helmet 18 LED group (marker group)
20 arithmetic control unit 21 search marker lighting unit 22 search marker association unit 23 search marker absolute coordinate calculation unit 24 search marker distance calculation unit 25 search marker identification unit 26 object position / angle identification unit 30 head motion tracker 40 Calculation control unit 41 All marker absolute coordinate calculation unit 42 All marker planned position calculation unit 43 All marker identification unit 44 New search marker absolute coordinate calculation unit 45 Object position / angle re-specification unit

Claims (7)

(A) A pair of cameras for viewing an object in stereo is installed,
(B) At least four lightable marker groups are attached to the outer surface of the object,
(C) determining the coordinates of each marker on the object fixed coordinate system defined on the object;
(D) Of the marker group, three markers that appear in the images of the pair of cameras at the same time, and the three markers that connect the three markers have a positional relationship that does not form a regular triangle are used as search markers. Lights up,
(E) For the three search marker images shown in each image, based on geometric prediction, the search marker images that are paired between the images are associated with each other,
(F) For the three search marker images associated with each other, calculate the absolute coordinates of each search marker on the absolute coordinate system defined on the space by triangulation using a pair of cameras;
(G) calculating the distance between the search markers from the absolute coordinates of each of the calculated three search markers;
(H) Based on the relationship between the distance between each marker obtained from the coordinates of each marker on the object fixed coordinate system and the calculated distance between the three search markers, Marker identification that identifies which marker is one of a group of markers attached to each object,
(I) An object characterized by specifying the position and angle of an object with respect to the absolute coordinate system based on the absolute coordinates of each of the three search markers specified by marker identification and the coordinates on the object fixed coordinate system. Monitoring method. Following the steps (a) to (i),
(J) calculating the absolute coordinates of all the markers based on the position and angle of the object with respect to the absolute coordinate system and the coordinates of each marker on the object fixed coordinate system;
(K) For all markers for which absolute coordinates have been calculated, calculate the planned positions where each marker appears on each image of a pair of cameras,
(L) The remaining markers excluding all markers or search markers are turned on, and marker identification for all markers is performed by associating a planned position on each image with a marker image position actually reflected,
(M) Among the identified markers, three markers that are simultaneously shown in the images of the pair of cameras are extracted as new search markers, and the absolute coordinates of these three new search markers are obtained by triangulation using the pair of cameras. To calculate
2. The position and angle of an object with respect to the absolute coordinate system are specified again based on the absolute coordinates of each of the three new search markers and the coordinates on the object fixed coordinate system. Object monitoring method. 2. The monitoring method according to claim 1, wherein in (a), the pair of cameras are installed so that a part of an object is viewed in stereo. In (b) above, the position on the object to which the marker group is attached is such that when a part of the object is projected with a pair of cameras, at least three markers of these marker groups are always reflected on each of the pair of cameras at the same time. As described above, the monitoring method according to claim 3, wherein the marker group is distributed over the entire outer surface of the object. 3. The monitoring method according to claim 2, wherein in (l), the association between the planned position and the marker image position is performed based on a distance between the two. Control computation for performing a stereo view of an object, at least four illuminable marker groups attached to the outer surface of the object, and controlling the pair of cameras and marker groups and computing the position and angle of the object A motion tracker with a
In the marker group, the coordinates of each marker on the object fixed coordinate system defined on the object are determined,
The control calculation unit searches for three markers in the marker group that are simultaneously displayed in the images of the pair of cameras, and in which the triangle connecting these three markers has a positional relationship that does not form an equilateral triangle. The three search marker images appearing in each image are associated with each other based on the geometric prediction, and the search marker images that are paired between the images are associated with each other. For the search marker image, calculate the absolute coordinates of each search marker on the absolute coordinate system defined in space by triangulation using a pair of cameras, and calculate the absolute coordinates of each of the calculated three search markers The distance between the markers for search is calculated from the distance between the markers obtained from the coordinates of each marker on the object fixed coordinate system determined previously and the three search markers calculated. Based on the relationship with the inter-car distance, the three search markers are identified from among a group of markers attached to the object, the markers are identified, and each of the three identified search markers is identified. A motion tracker characterized by performing an operation for specifying the position and angle of an object with respect to the absolute coordinate system based on the absolute coordinates of the object and the coordinates on the object fixed coordinate system. The control calculation unit further calculates the absolute coordinates of all the markers based on the position and angle of the identified object with respect to the absolute coordinate system and the coordinates of each marker on the object fixed coordinate system. For all the calculated markers, calculate the planned positions where each marker appears on each image of a pair of cameras, turn on the remaining markers excluding all markers or search markers, and calculate on each calculated image Markers are identified for all markers by associating the planned positions with the marker image positions that are actually reflected, and among the identified markers, three markers that appear simultaneously in each image of a pair of cameras are extracted as new search markers. Then, the absolute coordinates of these three new search markers are calculated from the relationship between the camera position and the new search marker image position displayed in the image. Search on the basis of the coordinates on the marker each absolute coordinates and the object fixed coordinate system, the motion tracker according to claim 6, characterized in that re-identifying the position and angle of the object relative to the absolute coordinate system.

Images