CN104035552A - Three-dimensional operation method and three-dimensional operation device - Google Patents

Abstract

The invention discloses a three-dimensional operation method and a three-dimensional operation device. The three-dimensional operation method includes that analyzing and judging the operation behavior of a user based on information measured by a distance sensor and an angle sensor arranged at the periphery of a screen; judging the operation type according to the judging result for the operation behavior of the user, and quantizing operation parameters; performing a three-dimensional operation on a user interface according to the operation type and operation parameters.

Description

Three-dimensional manipulating method and three-dimensional manipulating device

Technical field

The present invention relates to have the field of mobile terminals of three dimensional display, relate in particular to method and the three-dimensional manipulating device that can on three dimensional display, realize three-dimensional manipulating.

Background technology

Along with development and the application of three-dimensional (3D) technology, on mobile phone, also there is game and the application of a lot of 3D effects thereupon, screen can be shown 3D effect figure, makes user conscientiously experience three-dimensional display effect.

But existing touch-screen is all to need object directly to touch on screen, could realize some operation.Such as, need finger touches to screen, on screen, complete one section of sliding trace, judge user's behavior according to sliding trace, and then realize operation and control to mobile phone.

Due to the limitation of touch-screen, prior art is only suitable for operation and the control under 2D pattern, cannot solve displaying and control in 3d space.In other words, the plane touch screen of current this simple touch manner cannot complete the control function that stereoeffect is shown.

Summary of the invention

The present invention proposes in view of completing three-dimensional manipulating on touch-screen, does not touch screen even if its object is to provide a kind of, also can complete three-dimensional manipulating method and the three-dimensional manipulating device of the three-dimensional manipulating to three dimensional display.

Three-dimensional manipulating method of the present invention, comprises step: based on the measured range information of detection part and angle information analysis with judge user's operation behavior; According to the judged result of user's operation behavior and decision type, quantization operation parameter; According to action type and operating parameter, carry out three-dimensional manipulating in user interface.

And described detection part comprises three range sensors and angular transducer.

And described detection part further comprises that a range sensor and angular transducer are as check sensor.

And described detection part comprises radar.

And, described operating parameter comprise movement velocity, direction of motion and with screen angulation.

And described user's operation behavior is divided into one-way movement, Multidirectional motion and rotation.

And the user's of described one-way movement operation behavior judges as follows: whether the coordinate that judges the upper each point of finger variation tendency is at a time identical; Judge that it is linear in three coordinate planes, whether having a coordinate projection in plane at least; User's operation behavior is packaged into class.

And the user's of described Multidirectional motion operation behavior judges as follows: judgement finger number; Simulate the trail change of finger in space; User's operation behavior is packaged into class.

And the user's of described rotation operation behavior judges as follows: whether the gesture that tentatively judges user rotates; Judgement finger number; The rotation trend of analysis user gesture; Analysis user is pointed the movement locus in space; User's operation behavior is packaged into class.

Three-dimensional manipulating device of the present invention, comprises three dimensional display, controller, storer and detection part.

And described multiple distances and angular transducer are arranged on the periphery of described three dimensional display.

And described controller is according to the information analysis of described multiple distances and angular transducer and judge user's operation behavior, and judged user's operation behavior is referred to a kind of action type.

And described three dimensional display shows three-dimensional picture according to described action type.

By three-dimensional manipulating method provided by the present invention and three-dimensional manipulating device, can utilize the gesture in space to remove to operate three-dimensional picture, provide perfect support to 3D effect, allow UI become aestheticism more, more perceptual.

Brief description of the drawings

Fig. 1 is the process flow diagram of three-dimensional manipulating method of the present invention.

Fig. 2 is the schematic diagram of mounting distance and angular transducer on three dimensional display.

Fig. 3 is the three-dimensional coordinate system for certain any coordinate of obtaining on finger is described.

Fig. 4 is the process flow diagram of the deterministic process of the operation behavior of the user for one-way movement is described.

Fig. 5 is the process flow diagram of the deterministic process of the operation behavior of the user for Multidirectional motion is described.

Fig. 6 is the process flow diagram of the deterministic process of the user's formation for rotation behavior is described.

Embodiment

Below, the embodiment that present invention will be described in detail with reference to the accompanying.For convenience of explanation, mobile terminal of the present invention describes as an example of mobile phone example.Mobile phone of the present invention at least comprises three dimensional display, controller, storer and display.

First, with reference to figure 1, three-dimensional manipulating method of the present invention is described.

As shown in Figure 1, three-dimensional manipulating method of the present invention comprises:

Step 101: based on being separately positioned on the range sensor of screen periphery and angular transducer measured range information and angle information analysis and judging user's operation behavior.

Step 102: according to the judged result of user's operation behavior, decision operation type.

At this, user's operation behavior is summarized as different action types by software interface, then by relevant software interface offer user interface for.Here, action type mainly contain move, amplify, dwindle, rotation etc.

Step 103: according to action type, realize 3-D display in user interface.

Illustrate the method for the operation behavior that judges user below.

User's three-dimensional manipulating behavior has a variety of, if carry out quantitative analysis from the angle of changes in coordinates merely, roughly can user's operation behavior be divided into this three class of one-way movement, Multidirectional motion and rotation according to changes in coordinates.

In order to judge and the operation behavior of analysis user, the present invention has installed distance and angular transducer at three dimensional display four jiaos.Fig. 1 is illustrated in the schematic diagram of mounting distance and angular transducer on three dimensional display.As shown in Figure 1, at four jiaos of mounting distances and the angular transducer of three dimensional display, by sensor can measure on finger certain a bit and line and the three dimensional display angulation of the air line distance of sensor and this point and sensor.Also, can measure the information such as distance and angle of finger and sensor by range sensor, then calculate the multiple spot relative position of selling with three dimensional display by solid geometry algorithm, according to these information, user's operation behavior be judged.

Taking single-point as example, concrete computing method are as follows.

First suppose that mobile phone screen is in xy coordinate, the length of screen is the wide W of being of L.Set up as shown in Figure 2 coordinate system, certain on the finger some relative coordinate in coordinate system is d(x, y, z).According to the measurement data of sensor, we can measure the distance l of d and screen upper sensor ₁, l ₂, l ₃, and they and mobile phone screen angulation a, b, c.

Can list following equation according to above-mentioned measurement data:

z=l ₁sina………………1)

x ²+y ²=(l ₂cosb) ²………………2)

(W-x) ²+y ²=(l ₂cosa) ²………………3)

(L-y) ²+x ²=(l ₂cosc) ²………………4)

Just can calculate x according to aforesaid equation, y, the value of z is:

$x=\frac{W}{2}+\frac{{\left(l_2\cos b\right)}^2-{\left(l_1\cos a\right)}^2}{2W}---5)$

$y=\frac{L}{2}+\frac{{\left(l_2\cos b\right)}^2-{\left(l_3\cos c\right)}^2}{2L}---6)$

z=l ₁sina………………7)

Point the range information of each point and angle information only with three range sensors and angular transducer although measure user, but because may being subject to noise, environment etc. on measuring, sensor disturbs the certain error of generation, therefore the present invention is also provided with the 4th sensor as check sensor, according to the return data of the 4th sensor, utilize above-mentioned formula to recalculate, choose and measure coordinate points accurately, reach the effect of denoising.

Just can calculate the relative coordinate value D of the upper each point of finger and mobile phone screen according to method above ₁(x ₁, y ₁, z ₁), D ₂(x ₂, y ₂, z ₂) ... D _n(x _n, y _n, z _n), more just can depict the gesture of user in this moment according to these coordinates.User's finger is constantly moving, and the data in each moment are all different, use time ₁to time _mrepresent not in the same time, and use D ₁to D _nrepresent all reference point that are selected, so just can obtain one group of coordinate data below:

?	time 1	time 2	...	time M
					D 1	x 11,y 11,z 11	x 12,y 12,z 12	...	x 1M,y 1M,z 1M
D 2	x 21,y 21,z 21	x 22,y 22,z 22	...	x 2M,y 2M,z 2M
					D 3	x 31,y 31,z 31	x 31,y 31,z 31	...	x 3M,y 3M,z 3M
...	...	...	...	...
					D N	x N1,y N1,z N1	x N2,y N2,z N2	...	x NM,y NM,z NM

According to data above, just can carry out modeling to the coordinate figure in every bit or each moment, depict the variation track in space a little, then go to judge user's operation behavior according to the track of changes in coordinates.

Due to D ₁to D _ncoordinate be all three-dimensional spatial coordinate, in order to judge user's operation behavior according to changes in coordinates, coordinate need to be carried out to projection operation on three coordinate surfaces, the projection according to track on coordinate surface judges movement locus.Carry out quantitative analysis according to the different movement locus characteristics of motion separately again, finally determine user's behavior.

At this, D(x, y are found out in so-called projection exactly, z) to the vertical irradiation point of certain plane, for example, by D(x, y, z) three coordinate planes are carried out to projection operation, the subpoint coordinate obtaining is respectively d (x, y, 0), d (0, y, z), d (x, 0, z).

The method of the operation behavior that judges user is described with embodiment below.

Embodiment 1: one-way movement

One-way movement is the most a kind of movement tendency of application in existing touch screen technology, can be pointed the track sliding and completed by record the judgement of the operation behavior to user in existing touch screen technology on screen.But in the present invention, finger or palm are to slide in space, directly do not contact touch-screen, and existing judgment technology cannot realize.

In the present invention, the determination methods of the user's of one-way movement operation behavior as shown in Figure 4.

The feature of one-way movement is one or many situations of doing to be parallel to a certain coordinate surface or oblique movement operation with finger.In this case, all reference point are at time ₁to time _mmovement locus in time is unanimous on the whole.If to whole movement locus differential, choose the variation tendency at a time of coordinate a little carry out sample calculation, its rule meets following formula substantially:

$\frac{x_{M+1}-x_M}{y_{M+1}-y_M}\≈\frac{x_M-x_{M-1}}{y_M-y_{M-1}}$

$\frac{x_{M+1}-x_M}{z_{M+1}-z_M}\≈\frac{x_M-x_{M-1}}{z_M-z_{M-1}}$

$\frac{y_{M+1}-y_M}{z_{M+1}-z_M}\≈\frac{y_M-y_{M-1}}{z_M-z_{M-1}}$

Due to the speed difference that finger streaks, the variation tendency that can not ensure whole track is definite value, but differential is consistent to the variation tendency of single-point.

Therefore the coordinate that, first the determination methods of one-way movement of the present invention judges the upper each point of finger variation tendency whether identical (S401) at a time.

In addition, by D ₁to D _ncoordinate projection to three coordinate plane on time, on a coordinate plane, be at least linear.

Therefore, the determination methods of one-way movement of the present invention judges that in three coordinate planes, whether having a coordinate projection in plane at least is linear (S402) again.

Meet in the situation of above-mentioned 2 simultaneously, can judge that finger or palm are doing unidirectional linear movement, are packaged into class (S403) by user's operation behavior.

The present invention can also be according to the track of motion, the variation of coordinate, calculate the speed, direction of motion and with the operating parameters such as coordinate plane angulation, and the action type of the operation behavior corresponding to user and various operating parameter are offered to UI layer, show corresponding three-dimensional picture for UI layer.

Embodiment 2: Multidirectional motion

Multidirectional motion is also more common user's operation behavior, in existing touch screen technology, can judge by the track of the contact number in initial moment and touch the movement locus of hand, then according to track, user's behavior be judged.

But in three dimensions, the same with one-way movement, user's finger motion locus forms in three dimensions, does not touch touch-screen.

Fig. 5 is used for the process flow diagram of the deterministic process of the user's that Multidirectional motion is described operation behavior.

If judge user's operation behavior, first to judge and in the measurement range of screen, have several fingers (S501).

Judgement finger number can realize by the following method, that is, and and to time ₁moment coordinate a little carry out modeling, by time ₁the D in moment ₁to D _nall coordinates all project on three coordinate planes, the track now obtaining on each coordinate plane should be the linear track of many line-like, the number that the track number in the maximum plane of three coordinate plane neutral line tracks should be pointed exactly.Restore in space the position of finger in this moment by the track of subpoint again.

Then, simulate user and point the trail change (S502) in space.Can pass through time ₁to time _mthe D in moment ₁to D _ncoordinate carry out projection operation, the variation of coordinate points that just can be by coordinate plane, simulate the trail change in space.

Then, judge user's behavior according to trail change, as mode of motion such as finger polymerization, dispersions, user's operation behavior is packaged into class.

Embodiment 3: rotate

Rotation is the operation behavior of unexistent user in two-dimensional space, is the peculiar a kind of movement tendency of 3d space.Rotation is one changes in coordinates trend relatively freely.User's hand can rotate before screen with any posture taking wrist as axle.The movement locus of each reference point is similar one section of irregular arc all, cannot simply get on to judge that from single changes in coordinates trend hand is rotating.

Fig. 6 is the process flow diagram of the process for the rotation behavior that judges user's gesture is described.

First, the present invention tentatively judges user's gesture whether rotate (S601).Its determination methods is, to coordinate a little carry out Projection Analysis.Specifically, by time ₁to time _mthe D in moment ₁to D _ncoordinate (coordinate data a little) project to respectively on three coordinate surfaces, if now have at least a projected footprint in plane rounded or oval, just can tentatively judge user's the similar sphere of gesture, rotating.

Secondly, judgement finger number (S602).Utilize time ₁moment coordinate a little, determine finger number and shape within the scope of screen.

Then, the rotation trend (S603) of analysis user gesture.In the time that user's hand is rotated, time _m-1to time _min this process, on every finger, the variation of the vector of each point should be consistent.Method of vector analysis is as follows:

To time ₁to time _mthe D in moment ₁to D _ncoordinate analyze, calculate time ₁to time ₂, time ₂to time ₃..., time _m-1to time _mthe vector value of each point.

Postulated point D ₁and D ₂on same finger, 2 at time ₁to time ₂the variation of moment vector is identical, both meets can obtain by that analogy formula:

$\stackrel{\→}{D_{1M}{D}_{1M-1}}\×\stackrel{\→}{D_{2M}{D}_{2M-1}}=0$

In order to reduce measuring error, can carry out vector cross product to the multiple points on same finger and obtain the result after matching.In section, the sampling point vector variation tendency on each finger is identical at one time, can determine that user's gesture is rotating.

Then, analyze the movement locus of every finger in space, calculate rotation parameter (S604).

According to data above, can regard approx the variation tendency of user's gesture as a kind of similar sphere rotatablely move.Can calculate respectively like this time ₁to time _mthe information such as the centre of sphere, radius and the turning axle of moment spherical rotary, and calculate the vectorial variation tendency of each moment centre of sphere and turning axle.

Taking single point in time as example, introduce concrete computing method below:

The computing method > of the < centre of sphere and radius

Known theoretically, as long as there are 4 points on same circle just can obtain coordinate and the radius of the centre of sphere.Suppose that the radius of a ball is R, the approximate value of sphere centre coordinate is O (x ₀, y ₀, z ₀), the coordinate that the i on sphere is ordered is D _i(x _i, y _i, z _i), there is following relational expression in the coordinate of every bit and the radius of a ball:

(x ₀-x _i) ²+(y ₀-y _i) ²+(z ₀-z _i) ²=R ²………………8)

The coordinate of choosing like this four points both can calculate sphere centre coordinate value and radius.But this is theoretical value, because user's rotation action is similar spheric motion, the measurement of each point coordinate is chosen also certain error, so will carry out matching to the result of calculation of multiple points.

Here have a variety of methods available, for example, the most frequently used method is exactly exhaustive, all bring the coordinate of all observation point into equation and calculate multiple sphere centre coordinates, then carry out denoising matching, although this method is easily understood, but calculated amount is very large, affects handset capability.

Mainly introduce a kind of method of compensating computation below, in the hope of the reliable value of sphere centre coordinate and the radius of a ball.

To asking partial differential for sphere centre coordinate and the radius of a ball in formula above, and with its modified value δ _x, δ _y, δ _zand δ _rreplace differential, obtain:

2(x ₀-x _i)δ _x+2(y ₀-y _i)δ _y+2(z ₀-z _i)δ _z=2R ₀δ _R………………9)

At this, R ₀for radius match value.

$\frac{x_0-x_i}{R_0}{\mathrm{\&delta;}}_x+\frac{y_0-y_i}{R_0}{\mathrm{\&delta;}}_y+\frac{z_0-z_i}{R_0}{\mathrm{\&delta;}}_z={\mathrm{\&delta;}}_R---10)$

The observed reading R that the radius of a ball of supposing the approximate coordinate computation of the coordinate figure of ordering by i and the centre of sphere is the radius of a ball _t, its corrected value is V _rt,

$R_t=\sqrt{{(x_0-x_i)}^2+{(y_0-y_i)}^2+{(z_0-z_i)}^2}---11)$

R=R _t+V _Rt=R ₀+δ _R………………12)

V _Rt=R ₀-R _t+δ _R………………13)

In above formula, δ _rtry to achieve, can use formula substitution above, obtain the radius of a ball observed reading equation that i is ordered:

$V_{\mathrm{Rt}}=R_0-R_t+\frac{x_0-x_i}{R_0}{\mathrm{\&delta;}}_x+\frac{y_0-y_i}{R_0}{\mathrm{\&delta;}}_y+\frac{z_0-z_i}{R_0}{\mathrm{\&delta;}}_z---14)$

Equational general type is:

V _Rt=a _tδ _x+b _tδ _y+c _tδ _z+l _t………………15)

By equation and 3 unknown number δ of m observed reading _x, δ _y, δ _zthe equation of composition is

[aa]δ _x+[ab]δ _y+[ac]δ _z+[al]=0………………16)

[ab]δ _x+[bb]δ _y+[bc]δ _z+[bl]=0………………17)

[ac]δ _x+[bc]δ _y+[cc]δ _z+[cl]=0………………18)

Managing equational association factor matrices is

$Q=\left[\begin{array}{ccc}Q11& Q12& Q13\\ Q12& Q22& Q23\\ Q13& Q23& Q33\end{array}\right]$

The solution of unknown number is

δ _x=-[al]Q ₁₁-[bl]Q ₁₂-[cl]Q ₁₃………………19)

δ _y=-[al]Q ₁₂-[bl]Q ₂₂-[cl]Q ₂₃………………20)

δ _z=-[al]Q ₁₃-[bl]Q ₂₃-[cl]Q ₃₃………………21)

Coordinate after centre of sphere adjustment is:

x=x ₀+δ _x………………22)

y=y ₀+δ _y………………23)

z=z ₀+δ _z………………24)

The radius of a ball after adjustment is:

$R=R0-\frac{\left[l\right]}{m}$ ………………25)

In formula, molecule is observed reading equation constant term sum, the number that denominator is observed reading.

According to said method, just can comparatively accurately obtain the centre of sphere and the radius of the spherical rotary of single point in time, certain computing method have a variety of, and the present invention is only as example.

< turning axle computing method >

The rotating shaft of sphere is the straight line through the centre of sphere, and this axis is parallel with the normal vector of Plane of rotation.In other words, only requiring to obtain the normal vector of Plane of rotation, then substitution sphere centre coordinate, can determine unique turning axle.Now, how the normal vector of Plane of rotation calculates the key that has just become problem.

Here, so-called Plane of rotation refers to, at time ₁all point coordinate in moment revert to the shape that forms a similar sphere in coordinate system, choose the coordinate points D that approaches each finger fingertip ₁to D _tfor sampled point, what these sampled points can be similar to fits within same plane, and this plane is exactly the Plane of rotation in this moment.

Can determine straight line according to 2 in space, the straight line of two intersections just can be determined the principle of a plane, as long as there are not four points on same straight line just can determine a plane, and calculates his normal vector.

Supposing has D ₁to D _tindividual, be roughly dispersed in four to five regions, as long as choose respectively in zones of different, just can determine multiple Plane of rotations and calculate multiple normal vectors, these normal vectors are done to vectorial multiplication cross computing, if two planes are parallel, the result of vectorial multiplication cross is 0 again.In the present invention, can, to use the method for multidirectional amount matching, determine the normal vector of Plane of rotation.

Calculate after the information such as the centre of sphere, radius, turning axle of the simulation spheric motion of single point in time, just can use the same method to time ₂to time _mthe coordinate in moment carries out modeling, calculates the spherical rotary information in each moment, as shown in the table:

According to these information, just can further calculate the information such as centre of sphere change vector, the vector variation of turning axle, can do to the variation track of whole gesture the modeling analysis of a quantification.

Finally, user's operation behavior Information encapsulation is become to class, return to application programming interfaces.

Above-mentioned several users' operation behavior is just done simple a division from the angle of changes in coordinates, because the action of staff can be ever-changing, can also, according to the information such as speed and time of changes in coordinates, carry out finer analysis to movement locus.

In addition, although the present invention is illustrated as example to be arranged on distance on four jiaos of three dimensional displays and angular transducer, but the present invention is not limited thereto, the present invention can by be arranged on a radargrammetry user finger of mobile phone screen periphery certain a bit and the air line distance of radar and line and the three dimensional display angulation of this point and radar., the present invention can replace range sensor and angular transducer measuring distance information and angle information by radar.At this, the present invention does not limit the magnitude setting of radar, for example can two radars be set at mobile phone screen periphery, a radar is for measuring distance information and angle information, and another radar is for measured range information and the angle information accuracy of verification.

The present invention realizes the 3D of three dimensional display to control, and applies very extensive.Be used in the middle of the demonstration of 3D game and 3D effect figure, the operation behavior of the gesture analysis user by user, is packaged into unified UI interface, and application can utilize the UI style of these Interface realization different-effects, experiences to the diverse user of user.When the present invention is really applied on mobile phone, its application part is innumerable.For example:

1.3D cuts fruit: in space, do the action cutting fruit really by hand, complete game.

2. virtual magic square: user can directly do by gesture various rotation operations before mobile phone screen, and the magic square of directly controlling in screen forwards to, realizes 3D bandwagon effect.

3.TOM cat: user can do an action of drawing before mobile phone screen, removes to draw hair ear or the hair of TOM, to more expressing one's feelings and move TOM collocation in.

4.3D desktop: break original 2D desktop effect, the icon on desktop can be to make stereoeffect, and desk-top space also can become 3d space, by gesture, icon is drawn in, puts far, amplifies, and the operation such as dwindles.

The example of 3D operation is too numerous to enumerate, the invention has the advantages that and utilizes the gesture in space to go to judge user's behavior, is different from existing touch screen technology completely, and user just can complete operation without touching screen.And liberate finger, touching operation has been become to spatial movement behavioural analysis.User can have more selection, and more gesture removes to control mobile phone.Also for UI layer provides abundant class of operation, to the perfection support of 3D effect, allow UI become and obtain aestheticism more simultaneously, more perceptual.

Claims (13)

1. a three-dimensional manipulating method, is characterized in that, comprises step:

Based on the measured range information of detection part and angle information analysis with judge user's operation behavior;

According to the judged result of user's operation behavior and decision type, quantization operation parameter;

According to action type and operating parameter, carry out three-dimensional manipulating in user interface.

2. three-dimensional manipulating method according to claim 1, is characterized in that, described detection part comprises three range sensors and angular transducer.

3. three-dimensional manipulating method according to claim 2, is characterized in that, described detection part further comprises that a range sensor and angular transducer are as check sensor.

4. three-dimensional manipulating method according to claim 1, is characterized in that, described detection part comprises radar.

5. three-dimensional manipulating method according to claim 1, is characterized in that, described operating parameter comprise movement velocity, direction of motion and with screen angulation.

6. three-dimensional manipulating method according to claim 1, is characterized in that, described user's operation behavior is divided into one-way movement, Multidirectional motion and rotation.

7. three-dimensional manipulating method according to claim 6, is characterized in that, the user's of described one-way movement operation behavior judges as follows:

Whether the coordinate that judges the upper each point of finger variation tendency is at a time identical;

Judge that it is linear in three coordinate planes, whether having a coordinate projection in plane at least;

User's operation behavior is packaged into class.

8. three-dimensional manipulating method according to claim 6, is characterized in that, the user's of described Multidirectional motion operation behavior judges as follows:

Judgement finger number;

Simulate the trail change of finger in space;

User's operation behavior is packaged into class.

9. three-dimensional manipulating method according to claim 6, is characterized in that, the user's of described rotation operation behavior judges as follows:

Tentatively judge whether user's gesture rotates;

Judgement finger number;

The rotation trend of analysis user gesture;

Analysis user is pointed the movement locus in space;

User's operation behavior is packaged into class.

10. a three-dimensional manipulating device, is characterized in that, comprises three dimensional display, controller, storer and detection part.

11. three-dimensional manipulating devices according to claim 10, is characterized in that, described multiple distances and angular transducer are arranged on the periphery of described three dimensional display.

12. three-dimensional manipulating devices according to claim 10, it is characterized in that, described controller is according to the information analysis of described multiple distances and angular transducer and judge user's operation behavior, and judged user's operation behavior is referred to a kind of action type.

13. three-dimensional manipulating devices according to claim 12, is characterized in that, described three dimensional display shows three-dimensional picture according to described action type.