JP3953450B2 - 3D object posture operation method and program - Google Patents

Description

  The present invention relates to a method for manipulating the attitude of a three-dimensional object.

There is Patent Document 1 as a patent application related to the above technique, but in this patent application, only a three-dimensional position and direction indicated by an operator is detected, and a method for operating the posture of an object is described. It is not done. Moreover, although there exists patent document 2, since it is a matching system which used the hand image of various attitude | positions as a template, the attitude | position detected is discrete and calculation cost starts.
JP-A-10-326148 JP-A-8-076917

  Although it is possible in principle to detect the posture of a palm or fist and manipulate the posture of a three-dimensional object using this posture information, calculation is necessary to detect the posture of a palm or fist with sufficient accuracy. Expensive and real-time processing is impossible and impractical. Conversely, in order to enable real-time processing, the accuracy of posture detection such as palms and fists must be reduced. The degree of discreteness of the posture is large and impractical.

  An object of the present invention is to provide a three-dimensional object posture operation method and program for manipulating the posture of a three-dimensional object in a computer in real time.

According to the first aspect of the present invention, an apparatus having a position detection unit, a region determination unit, an operation mode determination unit, and an attitude calculation unit is configured so that a three-dimensional position of a predetermined object in real space A three-dimensional object posture operation method for manipulating the posture of a three-dimensional object, wherein the position detection means continuously detects the three-dimensional position of a predetermined object, and the three-dimensional position detected by the position detection means is fixed The area determination means determines whether the virtual work area is fixed inside the space coordinate system including the reference point, or is outside, and the operation mode determination means determines whether the operation mode determination means is a predetermined value from the determination result of the area determination means. When an object enters the virtual work area from the inside to the inside, the operation mode is changed from the idle mode to the posture operation mode, and when a predetermined object comes from the inside of the virtual work area to the outside, the operation mode is changed to the posture operation. From the reference mode to the three-dimensional position when entering the virtual work area of the predetermined object from the reference point. As long as the unit vector and the posture of the three-dimensional object are recorded and the posture operation mode is in effect, the posture calculation means moves the first unit vector from the reference point to the three-dimensional position of the current predetermined object. The rotation direction and rotation angle that are superimposed on the unit vector of 2 are subjected to rotation conversion on the three-dimensional object, and the posture of the three-dimensional object subjected to the rotation conversion is set as the current three-dimensional object posture.

According to the second aspect of the present invention, an apparatus having a position detection unit, a region determination unit, an operation mode determination unit, and an attitude calculation unit is configured so that a three-dimensional position of a predetermined object in real space A three-dimensional object posture operation method for manipulating the posture of a three-dimensional object, wherein the position detection means continuously detects the three-dimensional position of a predetermined object, and the three-dimensional position detected by the position detection means is fixed Inside or outside the virtual work area fixed in the space coordinate system, including the specified reference point, and inside or outside the internal virtual work area included in the virtual work area and fixed in the real space Is determined by the area determination means, and the operation mode determination means determines from the determination result of the area determination means that the operation mode is changed from the idle mode when a predetermined object enters the inside of the internal virtual work area. When a predetermined object comes out of the internal virtual work area from the inside of the virtual work area, the operation mode is changed from the posture operation mode to the idle mode, and at the transition from the idle mode to the posture operation mode. The operation mode determination unit records the first unit vector from the reference point toward the three-dimensional position when entering the internal virtual work area of the predetermined object, and the posture of the three-dimensional object. For a certain period of time, the posture calculation means performs rotation conversion of the rotation direction and rotation angle so as to superimpose the first unit vector on the second unit vector directed from the reference point to the current three-dimensional position of the predetermined object. The posture of the three-dimensional object that has been applied to the three-dimensional object and subjected to the rotation transformation is set as the current posture of the three-dimensional object .

According to the third aspect of the present invention, an apparatus having a position detection unit, a region determination unit, an operation mode determination unit, and a posture calculation unit is configured so that a predetermined object in a real space can be A three-dimensional object posture operation method for selecting one of three or more three-dimensional objects and manipulating the posture of the three-dimensional object, wherein the position detection means continuously detects the three-dimensional position of a predetermined object, The number of three-dimensional positions detected by the position detection means is the same as the number of three-dimensional objects, and corresponds to the three-dimensional object one-to-one, and is inside or outside each virtual work area fixed in real space. And the number of three-dimensional objects included in each virtual work area and fixed in the real space, and inside each non-intersecting internal virtual work area The area determination means determines whether the area is outside, and the operation mode determination means determines from the determination result of the area determination means that the predetermined object is the first 3 from outside the virtual work area of all three-dimensional objects. When entering the inside of the internal virtual work area of the three-dimensional object, the operation mode is changed from the idle mode to the posture operation mode of the three-dimensional object, and the predetermined object is virtualized from the inside of the internal virtual work area of the three-dimensional object. When moving outside the work area, the operation mode is changed from the posture operation mode to the idle operation mode, and at the time of transition from the idle mode to the posture operation mode, the operation mode determination means is fixed in the real space for the three-dimensional object. A first unit vector from the reference point toward the three-dimensional position when entering the internal virtual work area of the predetermined object; Only when the posture operation mode is in the posture operation mode, the posture calculation means sends the first unit vector from the reference point to the current three-dimensional position of the predetermined object. The rotation direction and rotation angle that are superimposed on the vector are subjected to rotation conversion on the three-dimensional object, and the posture of the three-dimensional object subjected to the rotation conversion is set as the current three-dimensional object posture.

  According to the present invention, by moving the position of a fingertip or the like, it is possible to easily specify gripping and releasing of a virtual object or to easily operate the posture of the virtual object.

Next, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]

  In the present embodiment, by analyzing a camera image obtained by imaging a marker attached to the fingertip, the three-dimensional movement of the fingertip in the space is captured, and the posture of the three-dimensional object in the computer is manipulated. 1 to 5 show a method for detecting the three-dimensional position of the fingertip in the space.

  In FIG. 1, the camera 104 and the camera 105 are fixed to a spatial coordinate system 190, and the central axes of the cameras 104 and 105 intersect at a reference point 102 on the Z axis of the spatial coordinate system 190. The display monitor 103 is placed between the camera 104 and the camera 105. The marker 101 attached to the fingertip is a sphere painted in fluorescent red, but a light emitting diode having a wavelength of about 900 nanometers can also be used as a marker.

  In FIG. 2, images 200 and 201 are images obtained by processing the entire red channel images of the cameras 104 and 105 capturing the fingertip marker 101 with a Gaussian filter. The fingertip marker images 202 and 203 are displayed as high brightness areas.

  In FIG. 3A, the camera coordinate system 110 is fixed to the spatial coordinate system 190, and its Z axis coincides with the central axis of the camera 104. A straight line 113 is a straight line that connects the position of the pixel 112 with the highest luminance on the image plane 111 and the origin of the camera coordinate system 110. In FIG. 3B, the camera coordinate system 120 is fixed to the spatial coordinate system 190, and its Z axis coincides with the central axis of the camera 105. A straight line 123 is a straight line that connects the position of the pixel 122 with the highest luminance on the image plane 121 and the origin of the camera coordinate system 120.

  FIG. 4 shows a straight line 113 and a straight line 123 in the spatial coordinate system 190. Since the camera coordinate system 110 and the camera coordinate system 120 are fixed to the spatial coordinate system 190, the straight line 113 and the straight line 123 whose positions and orientations are specified in the respective camera coordinate systems 110 and 120 are the same in the spatial coordinate system 190. The location is also specified.

FIG. 5 shows a method for detecting the three-dimensional position of the fingertip in real space. Intersections between the straight line 113 and the straight line 123 whose positions in the spatial coordinate system 190 are identified in the shortest and the straight line 113 and the straight line 123 are defined as P ₁ and P ₂ , respectively. A midpoint P _M between P ₁ and P _{2 is set} as a three-dimensional position 100 of the fingertip in the real space.

  FIGS. 6 to 11 show a method of operating a three-dimensional object in the computer by the three-dimensional position of the fingertip in the real space.

  FIG. 6 shows a virtual work area 130 fixed to the spatial coordinate system 190 so as to include the reference point 102.

  FIG. 7 shows a state of the idle mode in which the operation mode is before the start of operation. FIG. 7A is a monitor screen on which a three-dimensional object 210 to be operated, a cursor 211 indicating the fingertip position, and an image 212 of the boundary 131 of the virtual work area 130 are displayed. FIG. 7B shows a state where the three-dimensional position 100 of the fingertip is outside the virtual work area 130.

  FIG. 8 shows a state in which the operation mode is changed to the posture operation mode when the three-dimensional position 100 of the fingertip outside the virtual work area 130 enters the virtual work area 130. At this time, the unit vector 141 from the reference point 102 shown in FIG. 10 toward the contact point 140 between the three-dimensional position 100 of the fingertip and the boundary surface 131 of the virtual work area 130 and the posture of the three-dimensional object 210 are stored. deep. At this time, as a method of notifying the operator of the transition of the operation mode, the display color of the three-dimensional object 210 is changed, the display color of the image 212 of the boundary 131 and the cursor 211 of the virtual work area 130 is changed, or a chime is sounded. There is a way. The posture operation mode is continued while the three-dimensional position 100 of the fingertip is inside the virtual work area 130 or on the boundary 131.

  FIG. 9 shows a state during posture operation. Regarding the current posture of the three-dimensional object 210, the posture of the three-dimensional object 210 stored when the motion mode is changed to the posture operation mode is indicated by 144 around the axis 143 shown in FIG. The posture rotated by an angle is taken as the current posture. Here, the axis 143 is a straight line perpendicular to a plane including both the unit vector 141 shown in FIG. 10A and the unit vector 142 from the reference point 102 toward the three-dimensional position 100 of the fingertip and passing through the reference point 102. It is. An angle indicated by 144 is an angle formed by the unit vector 142 with respect to the unit vector 141.

FIG. 11 shows a state where the posture operation is completed when the three-dimensional position 100 of the fingertip comes out of the virtual work area 130 and the operation mode changes from the posture operation mode to the idle mode. This state is the same as the state shown in FIG.
[Second Embodiment]

  In the first embodiment, at the time of the operation mode transition from the idle mode to the posture operation mode described in FIG. 8 or at the time of the operation mode transition from the posture operation mode to the idle mode described in FIG. Chattering may occur when noise or the like overlaps with the detected three-dimensional position 100 of the fingertip.

  As a second embodiment, FIGS. 12 to 16 show a method of operating a three-dimensional object in a computer based on the three-dimensional position of a fingertip in real space while preventing chattering.

  FIG. 12 shows a state where the operation mode is the idle mode. As shown in FIG. 12B, in addition to the virtual work area 130 shown in FIG. 7, an internal virtual work area 150 included therein is used. It is assumed that the minimum value of the distance between the boundary surface 131 and the boundary surface 151 of both virtual work areas 130 and 150 is sufficiently larger than the level of noise superimposed on the three-dimensional position 100 of the fingertip. In the display monitor 103 shown in FIG. 12A, the image 215 of the boundary surface 131 of the virtual work area 130 is not displayed or is displayed in a light color, while the image 213 of the boundary surface 151 of the internal virtual work area 150 is displayed. Is displayed in dark color and informs the operator of the boundary surface to be tilted.

  FIG. 13 shows a state of transition from the operation mode idle mode to the posture operation mode. Only in the idle mode, when the three-dimensional position 100 of the fingertip outside the internal virtual work area 150 enters the internal virtual work area 150 or comes into contact with the boundary surface 151, the operation mode transitions to the posture operation mode. At that time, the unit vector 141 from the reference point 102 shown in FIG. 15 toward the contact point 140 between the three-dimensional position 100 of the fingertip and the boundary surface 131 of the virtual work area 130 and the posture of the three-dimensional object 210 are stored. deep. At this time, the display color of the cursor 211 is changed, the chime is sounded, or the image 213 of the boundary surface 151 of the internal virtual work area 150 is deleted or changed to a light color display, while the boundary surface 131 of the virtual work area 130 is changed. By changing the image 215 to a dark color display, the operator is notified of the transition of the operation mode, and the boundary surface to be tilted is also notified. While the three-dimensional position 100 of the fingertip is inside the virtual work area 130, the posture operation mode is continued.

  FIG. 14 shows a state during the posture operation. Regarding the current posture of the three-dimensional object 210, the posture of the three-dimensional object 210 stored when the operation mode and the posture operation mode are changed is indicated by 144 around the axis 143 shown in FIG. The posture rotated by an angle is taken as the current posture. Here, the axis 143 is a straight line perpendicular to a plane including both the unit vector 141 shown in FIG. 15A and the unit vector 142 from the reference point 102 toward the three-dimensional position 100 of the fingertip and passing through the reference point 102. It is. An angle indicated by 144 is an angle formed by the unit vector 142 with respect to the unit vector 141.

FIG. 16 shows a transition of the operation mode from the posture operation mode to the idle mode. Only in the posture operation mode, when the three-dimensional position 100 of the fingertip comes out of the virtual work area 130 or comes into contact with the boundary surface 131, the mode changes to the idle mode.
[Third Embodiment]

As shown in FIG. 17, instead of the internal virtual work area 150 in the second embodiment, an internal virtual work area 160 that is an area having the same shape as the three-dimensional object 210 is used.
[Fourth Embodiment]

In the fourth embodiment, a case will be described in which there are two virtual work areas similar to the virtual work areas shown in FIG. 12 of the second embodiment, and these virtual work areas intersect.

As shown in FIG. 18, the intersection area of the two internal virtual workspace, the area removed from the respective internal virtual workspace by respectively an internal virtual workspaces 150 and the internal virtual workspace 550, the second The posture of the three-dimensional object is manipulated by the same method as in the embodiment.

  In the first embodiment, a detection engine using a marker is used as means for detecting the three-dimensional position of the fingertip. However, instead of this engine, an engine for detecting the position of the fingertip without anything is attached. It can also be used. As such a fingertip three-dimensional position detection engine, there is “Robust Finger Tracking with Multiple Cameras”, Cullen Jennings, 1999, and the like.

  FIG. 19 is a block diagram of an apparatus that implements the three-dimensional object posture operation methods of the first to fifth embodiments described above.

  The fingertip or marker position detection unit 301 detects the three-dimensional position 100 of the fingertip or the marker 101 attached to the fingertip. The area determination unit 302 determines whether the three-dimensional position 100 of the fingertip or the marker 101 is located outside the virtual work area 130 (in addition to the internal virtual work area 150 in the second embodiment) or the virtual work area 130. . The operation mode determination unit 304 determines the operation mode (idle mode, posture operation mode) from the determination search of the region determination unit 302, stores the determined operation mode in the operation mode storage unit 303, and the operation mode changes from the idle mode to the posture. When transitioning to the operation mode, the unit vector 41 and the posture of the three-dimensional object 210 are stored in the position information storage unit 305. The posture calculation unit 306 performs calculation for rotating the posture of the three-dimensional object 210 around the axis 143 by an angle 144 using the information stored in the position information storage unit 305 when the operation mode is the posture operation mode.

  The three-dimensional object posture operation method of the present invention described above records a program for realizing the function on a computer-readable recording medium, and reads the program recorded on the recording medium into a computer system. Can be executed. The computer-readable recording medium refers to a recording medium such as a floppy disk, a magneto-optical disk, a CD-ROM, or a storage device such as a hard disk device built in the computer system. Furthermore, a computer-readable recording medium is a server that dynamically holds a program (transmission medium or transmission wave) for a short period of time as in the case of transmitting a program via the Internet, and a server in that case Some of them hold programs for a certain period of time, such as volatile memory inside computer systems.

It is a figure which shows a mode that the fingertip marker 101 is image | photographed with two cameras 104,105. It is a figure which shows the image which processed the whole red channel image of the cameras 104 and 105 which image | photographed the fingertip marker 101 by the Gaussian filter. It is a figure which shows the straight lines 113 and 123 which connect the image 112 and 122 with the highest brightness | luminance on the image planes 111 and 121, and the origin of the camera coordinate system 120. FIG. It is a figure which shows the straight lines 113 and 123 in the spatial coordinate system 190. FIG. It is a figure which shows the method of detecting the three-dimensional position of the fingertip in real space. FIG. 4 is a diagram showing a virtual work area 130 including a reference point 102. It is a figure which shows the state whose operation mode is an idle mode. It is a figure which shows the state which operation mode changed to attitude | position operation mode. It is a figure which shows the state during attitude | position operation. FIG. 6 is a diagram illustrating a change in posture of a three-dimensional object 210. It is a figure which shows the state which operation mode changed from attitude | position mode to idle mode, and attitude | position operation was completed. It is a figure which shows the idle mode in 2nd Embodiment. It is a figure which shows the mode of the operation mode transition from idle mode in 2nd Embodiment to attitude | position operation mode. It is a figure which shows the state during attitude | position operation in 2nd Embodiment. It is a figure which shows the change of the attitude | position of the three-dimensional object 210 in 2nd Embodiment. It is a figure which shows the mode of the operation mode transition from the attitude | position operation mode in 2nd Embodiment to idle mode. It is a figure which shows the internal virtual work area used by 3rd Embodiment. It is a figure which shows the virtual work area used in 4th Embodiment. It is a block diagram of the apparatus which implements the three-dimensional object attitude | position operation method of 1st-5th embodiment.

Explanation of symbols

100 Estimated Marker Position in Real Space 101 Marker Position in Real Space 102 Attitude Operation Reference Point on Z-axis of Real Space Coordinate System 103 Display Monitor 104 Camera 105 Camera 190 Real Space Coordinate System 200 Camera Image 201 Camera image 202 Marker position on camera image 203 Marker position on camera image 110 Camera coordinate system 111 Camera imaging surface 112 Marker position on camera imaging surface 113 Origin of camera coordinate system and marker position on camera imaging surface 120 Camera coordinate system 121 Camera imaging plane 122 Marker position on camera imaging plane 123 Straight line connecting origin of camera coordinate system and marker position on camera imaging plane 130 Virtual work area 131 Boundary of virtual work area Surface 132 Virtual work area 151 Internal virtual work area Boundary surface 210 Three-dimensional object displayed on monitor 211 Cursor at marker position 212 Virtual work area displayed on monitor 140 Contact point between estimated marker position and boundary surface of virtual work area 141 From posture operation reference point to contact point Heading unit vector 142 Unit vector heading to the marker position estimated from the posture operation reference point 143 Straight line passing through the posture operation reference point perpendicular to the plane including the two unit vectors 144 Angle formed by the two unit vectors 108 Three-dimensional object 109 3 Dimensional object 301 Fingertip or marker position detection unit 302 Area determination unit 303 Operation mode storage unit 304 Operation mode determination unit 305 Position information storage unit 306 Posture calculation unit

Claims (7)

Position detecting means, region determination means, the operation mode determining means, and a device having the orientation calculation unit, the three-dimensional position of the predetermined object in the real space, three-dimensional for operating the attitude of the calculation machine of the three-dimensional object An object posture operation method,
The position detecting means continuously detects the three-dimensional position of the predetermined object;
The area determination means determines whether the three-dimensional position detected by the position detection means is inside or outside a virtual work area fixed in a spatial coordinate system including a fixed reference point. ,
When the predetermined object enters the inside of the virtual work area from outside the virtual work area based on the determination result of the area determination means, the operation mode determination means changes the operation mode from the idle mode to the posture operation mode, and the predetermined object When the operation mode is moved from the inside of the virtual work area to the outside, the operation mode is changed from the posture operation mode to the idle mode,
At the time of transition from the idle mode to the posture operation mode,
The operation mode determination means records a first unit vector from the reference point toward the three-dimensional position when entering the virtual work area of the predetermined object, and the posture of the three-dimensional object,
Only while in the posture operation mode,
Rotation conversion of a rotation direction and a rotation angle such that the posture calculation means superimposes the first unit vector on a second unit vector directed from the reference point toward the current three-dimensional position of the predetermined object. A three- dimensional object posture operation method , wherein the posture of the three-dimensional object applied to the three-dimensional object and subjected to the rotation transformation is set as the current posture of the three-dimensional object. Position detecting means, region determination means, the operation mode determining means, and a device having the orientation calculation unit, the three-dimensional position of the predetermined object in the real space, three-dimensional for operating the attitude of the calculation machine of the three-dimensional object An object posture operation method,
The position detecting means continuously detects the three-dimensional position of the predetermined object;
Whether the three-dimensional position detected by the position detecting means is inside or outside a virtual work area fixed in a spatial coordinate system including a fixed reference point, and is included in the virtual work area; The area determination means determines whether the internal virtual work area fixed in the real space is inside or outside, and
The operation mode determination means changes the operation mode from the idle mode to the posture operation mode when the predetermined object enters the inside of the internal virtual work area from the determination result of the area determination means, and When an object comes out of the virtual work area from the inside of the internal virtual work area, the operation mode is changed from the posture operation mode to the idle mode,
At the time of transition from the idle mode to the posture operation mode,
The operation mode determination means records a first unit vector from the reference point toward the three-dimensional position when entering the internal virtual work area of the predetermined object, and the posture of the three-dimensional object,
Only while in the posture operation mode,
Rotation conversion of a rotation direction and a rotation angle such that the posture calculation means superimposes the first unit vector on a second unit vector directed from the reference point toward the current three-dimensional position of the predetermined object. A three- dimensional object posture operation method , wherein the posture of the three-dimensional object applied to the three-dimensional object and subjected to the rotation transformation is set as the current posture of the three-dimensional object. The three-dimensional object posture operation method according to claim 2, wherein an area formed by the shape of the three-dimensional object is used as the internal virtual work area. An apparatus having a position detection unit, a region determination unit, an operation mode determination unit, and an attitude calculation unit selects one of two or more three-dimensional objects in a computer according to the three-dimensional position of a predetermined object in real space. , A three-dimensional object posture operation method for manipulating the posture of the three-dimensional object ,
The position detecting means continuously detects the three-dimensional position of the predetermined object;
The three-dimensional positions detected by the position detecting means are the same as the number of the three-dimensional objects, and correspond to the three-dimensional objects on a one-to-one basis , in each of the virtual work areas fixed in the real space. Whether it is inside or outside each of the internal virtual work areas that are included in each virtual work area and are fixed in the real space, the same number as the number of the three-dimensional objects, that do not intersect each other. The area determination means determines whether it is outside,
From the determination result of the area determination means, the operation mode determination means determines that the predetermined object is outside the virtual work area of all the three-dimensional objects and inside the internal virtual work area of the first three-dimensional object. When entering, the operation mode is changed from the idle mode to the posture operation mode of the three-dimensional object, and the predetermined object comes out of the virtual work area from the inside of the internal virtual work area of the three-dimensional object. When the operation mode is changed from the posture operation mode to the idle mode,
At the time of transition from the idle mode to the posture operation mode,
A first unit vector directed to a three-dimensional position when the operation mode determination means enters the internal virtual work area of the predetermined object from a reference point fixed in the real space with respect to the three-dimensional object; , Record the posture of the three-dimensional object,
Only while in the posture operation mode,
Rotation conversion of a rotation direction and a rotation angle such that the posture calculation means superimposes the first unit vector on a second unit vector directed from the reference point toward the current three-dimensional position of the predetermined object. A three- dimensional object posture operation method , wherein the posture of the three-dimensional object applied to the three-dimensional object and subjected to the rotation transformation is set as the current posture of the three-dimensional object. As the internal virtual work area, an area formed by the shape of a three-dimensional object corresponding to the internal virtual work area, an area formed by all three-dimensional objects other than the three-dimensional object, and a shape of the three-dimensional object are formed. The three-dimensional object posture operation method according to claim 4, wherein the area from which the intersection with the area is removed is used.   The three-dimensional object posture operation method according to claim 1, wherein a fingertip or a marker is used as the predetermined object. A three-dimensional object posture operation program for causing a computer to execute the three-dimensional object posture operation method according to any one of claims 1 to 6 .

Images