CN106527738A - Multi-information somatosensory interaction glove system and method for virtual reality system - Google Patents

Abstract

The invention provides a multi-information somatosensory interaction glove system and method for a virtual reality system, and relates to the technical field of virtual reality. The somatosensory interaction glove system comprises a glove body, a sensor module, a signal operation amplifier module, an analogue-to-digital conversion module, a vibration feedback module, a processor module and a communication module. Hand motion information and attitude calculation is carried out by adopting the somatosensory interaction glove system, a computing method is loaded into the processor module of the glove system in a program form, hand motion and attitude information is computed in real time, and meanwhile, feedback information of an upper computer of the virtual reality system is received and the vibration feedback module is controlled to simulate touch, thereby achieving bidirectional interaction. Through a multi-sensor information fusion technology and sEMG-based implicit interaction, accurate recognition and computing can be carried out on a three-dimensional motion attitude and force of a hand of an operator, and an immersive interaction experience is brought for a user.

Description

A kind of multi information body feeling interaction glove system and method for virtual reality system

Technical field

The present invention relates to technical field of virtual reality, more particularly to a kind of multi information body-sensing friendship for virtual reality system Mutual glove system and method.

Background technology

The big puzzlement that the information input mode problem of virtual reality system is always studied, and for hand exercise information This topmost virtual reality input, existing interactive glove are examined by single Inertia information or bend sensor Survey hand gestures, it is impossible to real accurate simulation, the size of particularly unpredictable motoricity, real experiences are carried out to hand motion It is unnatural；And existing interactive glove do not have feedback interactive function, simply system on human carries out unidirectional gesture identification, this Plant interactive mode and cannot bring the virtual experience for truly immersing.

The content of the invention

For the defect of prior art, the present invention provides a kind of multi information body feeling interaction glove for virtual reality system System and method, input of the multi information body feeling interaction glove system as virtual reality system, by multi-sensor information fusion Technology and the implicit interactions based on sEMG, can accurately be recognized and be counted to the hand three-dimensional motion attitude of operator and power Calculate, be that user brings interactive experience on the spot in person.

On the one hand, the present invention provides a kind of multi information body feeling interaction glove system for virtual reality system, including handss Set body, sensor assembly, signal operation amplifier module and analog-to-digital conversion module, vibrational feedback module, processor module and Communication module.

Glove bulk is used to carrying and fixing other module groups, is worn on user hand, including finger section, wrist portion and Oversleeve portion, oversleeve portion can be covered to user fore-arm.

Sensor assembly, for gathering motion and the attitude information of hand, including bend sensor group, myoelectric sensor Group, inertial sensor group and pressure transducer group；Bend sensor group is fixed on the outside of each fingerstall of glove bulk finger section In finger portion interlayer, for gathering digital flexion degree, including 5 flexible bending sensors；Myoelectric sensor group is fixed on glove sheet The forearm oversleeve portion of body, for gathering the skin surface myoelectric information (sEMG) that the motion of hand exercise related muscles is produced, including 8 Individual myoelectric sensor；Inertial sensor group is fixed on the wrist portion of glove bulk, for gathering the movable information at wrist, including Three-dimensional accelerometer and three-dimensional gyroscope；Pressure transducer group is fixed on the inside of each fingerstall of glove bulk finger section, is used for Measurement finger contact force, including 5 contact pressure sensors.

Signal operation amplifier module, is fixed on the outside of glove bulk forearm oversleeve portion, for myoelectric sensor group is adopted The sEMG micro-signals of collection are amplified.

Analog-to-digital conversion module, is fixed on the outside in glove bulk forearm oversleeve portion, for what is detected myoelectric sensor group Analogue signal after signal operation amplifier amplifies is processed and is converted into digital signal, to improve adopting for electromyographic signal Sample rate.

Vibrational feedback module, is fixed on the inside of the fingerstall of glove bulk finger section and locates, for by the handss of virtual reality system Portion's tactile data feeds back to hand, including 4 vibrational feedback units in the way of shaking.

Processor module, is fixed on the outside of the wrist portion of glove bulk, for collecting and processing the letter of analog-to-digital conversion module Number, and calculate attitude, displacement, contact force and the contactless force of hand, and the hand tactile data for receiving virtual reality system And feed back to vibrational feedback module.

Communication module, is fixed on by processor module, for connecting the host computer of processor module and virtual reality system, Communication for information and communication are realized, the communication module is bluetooth communication module.

On the other hand, the present invention also provides a kind of using the above-mentioned multi information body feeling interaction handss for virtual reality system Set system carries out the computational methods of hand exercise information and pose, and the method is loaded into above-mentioned body feeling interaction glove in the form of program The processor module of system, is calculated to hand exercise and attitude information in real time, while receiving virtual reality system host computer Feedback information and control vibrational feedback module to simulate tactile, realize two-way interactive, concrete grammar is as follows：

Step 1：The signal of each sensor group in real-time collecting sensor assembly, and carry out analog digital conversion；

Step 2：Design digital filter algorithm, to analog digital conversion after each digital signal be filtered and denoising is located in advance Reason；

Step 3：Each finger camber is calculated according to the data of bend sensor group collection, according to the number of myoelectric sensor group According to the activity for calculating finger related muscles, according to the three-dimensional accelerometer signal and three-dimensional gyroscope signal of inertial sensor group The three-dimensional perspective of wrist linear movement and the anglec of rotation, i.e. hand gestures is calculated, according to the data acquisition handss of pressure transducer group Abutment power；

Step 4：Many muscle kinetic models are set up, Fingers power is estimated, refer to that power includes contact force and noncontact Power, specifically includes following steps：

Step 4.1：Carry out the finger power based on many muscle kinetic models to predict, set up between muscle activation degree and muscular force Many muscle kinetic models, for calculating muscular force by the related muscles activity that is input into, the muscular force is used for predicting finger Refer to power；

Step 4.2：The finger that the finger contact force detected using pressure transducer group is estimated with many muscle kinetic models Refer to that power sets up state equation, Fingers power is accurately estimated by EKF；

Step 5：The finger camber for obtaining, wrist linear movement and the anglec of rotation, finger contact force and contactless force are believed Breath is sent to communication module port, while the tactile feedback information for receiving virtual reality system host computer is used to control vibrational feedback Module.

Further, in the step 2, the preprocessing process of sEMG signals is carried out using IIR digital band-pass filters Denoising, mainly filters high-frequency noise and 50Hz Hz noises, computing formula of the formula (1) for preprocessing process；

Wherein, L represents bandpass filter function；E (t) is original sEMG signals；For average in access time window Value；U (t) is pretreated sEMG signals；u₀For side-play amount.

Further, in the step 3, the computational methods of wrist linear movement and the anglec of rotation are：

Definition rotational steps are p, and the rotational steps are the sum of the angular velocity absolute value in T sampled point, are shown below：

Wherein, abs () is to ask signed magnitude arithmetic(al),For the axial angular velocity of t-th sample point i；

Two kinds of method for solving of hand gestures (three-dimensional perspective) are defined, is calculated by accelerometer and gyro data respectively and is obtained ：

Method 1：Joint angles are calculated using acceleration transducer：

Determine vector R of three directional acceleration signals of x, y, z, be shown below：

Wherein, a_x、a_y、a_zThe respectively acceleration signal in three directions of x, y, z；

Determine angle angle of all directions_i, as shown in formula (4), wherein, i=x, y, z.

Method 2：Joint angles are calculated using gyroscope, as shown in formula (5)；

angle_i(n)=angle_i(n-1)+w_i·T₁ (5)

Wherein, angle_iThe joint angles of (n) for current sampling point, angle_i(n-1) be previous sampled point joint angle Degree, w_iFor the angular velocity in the i directions of collection, T₁For the time rate of change of gyroscope gathered data；

As rotational steps p ＜ p₁When, it is static or lower-speed state, using method 1 calculates hand three-dimensional perspective；Work as p₁≤p≤ p₂When, speed is moderate, and the result weighted sum of using method 1 and 2 calculates hand three-dimensional perspective；As p ＞ p₂When, speed is used Method 2 calculates hand three-dimensional perspective；Wherein, p₁、p₂And p₃Respectively 20%, 40% and the 60% of hand exercise maximum angular rate.

Further, the step 4.1 carries out the finger power prediction based on many muscle kinetic models, specifically includes following step Suddenly：

Step 4.1.1：Activity model is set up, for calculating the activation degree of muscle, the size of muscular force, muscle is reacted Activity is calculated according to formula (6) by pretreated sEMG signals；

Wherein, a (t) represents the activity of t, and u (t) is the pretreated sEMG signals of t；

Step 4.1.2：Muscle contraction kinetic model, abbreviation Hill models are set up, the model includes producing actively contraction PowerActive contractile unit (CE) and the passive elastic force of generation connected in parallelFlexible element (PE)；

Tendon units power F in Hill models₁ ^mtBy active contractilityWith passive elastic forceSuperposition F^mProduce, such as Shown in following formula；

Wherein, φ is the angle of meat fiber and tendon；Active contractilityBy itself and muscle length item functionWith Muscle contraction speed term function f_VThe flesh contractility such as (v), muscle activation degree a (t) and muscle maximumRepresent, passive elastic forceBy itself and muscle length item functionWith the flesh contractility such as muscle maximumRepresent, as shown in formula (8)；

It is approximated as follows hypothesis：The included angle of meat fiber and tendon keeps constant, takes 0.2；Muscle contraction speed is to actively The impact of power change can be ignored；The ratio of rigidity of tendon is larger, i.e. tendon length is held essentially constant；

Then normalization muscle length exhibitionCan be calculated by following formula：

Wherein, l^mRepresent muscle length,Represent optimal muscle length；

Active contractility and muscle length item functionWith passive elastic force and muscle length item functionLetter Number form formula, by the existing data based on medical research as curve matching setting up, resulting fitting function formula is respectively such as Shown in formula (10) and formula (11)；

Step 4.1.3：By muscle activation degree a (t) for obtaining and normalization muscle lengthFormula (7) is brought into formula (9), Obtain muscle force prediction value.

Further, in the step 4.2, the method for the accurate estimation of EKF is：

Step 4.2.1：Set up the state equation with Fingers power as state；

If Fingers power F^mtWith the derivative of the powerFor the state of Kalman filtering, the Fingers power of n-th sampled point and Shown in the state of its derivative such as formula (12) and formula (13)；

Wherein, w₁N () is the process noise in Fingers power state description equation, w₂N () is described for finger strength derivative state Process noise in equation, T are the sampling time；

The state vector of n-th sampled point be x (n), estimated value F of Fingers power^cAnd its derivativeComposition output vector ForFormula (12) and formula (13) simultaneous obtain state equation as follows：

Wherein, z (n) is the observer state for referring to power, is determined by measured value, process noises of the w (n) for model, and v (n) is survey Amount noise, is white Gaussian noise, i.e. w=N (0, Q), v=N (0, R), wherein, Q and R are constant；NoteH= [1 1]；

Step 4.2.2：Calculation error matrix simultaneously updates state equation；

Step 4.2.2.1：The least mean-square error square of current sampling point is calculated by formula (15) by the value of a upper sampled point Battle array P (n | n-1), P (n-1 | n-1) are the least mean-square error matrix of a upper sampled point；

P (n | n-1)=A P (n-1 | n-1) A^T+Q (15)

For first sampled point, initialization least mean-square error matrix P (1 | 1) is unit battle array；

Step 4.2.2.2：Least mean-square error matrix according to obtaining is updated amendment to current state, such as formula (16) It is shown, least mean-square error matrix P including calculation error gain K (n), correction value x (n | n) and current state (n | n)；

Wherein, x (n | n-1) represents the current state value calculated under the conditions of preceding state is；

Step 4.2.2.3：By the state of formula (16) newer (14) state equation, the finger strength of each sampled point is obtained Estimated value, i.e. state equation are exported.

As shown from the above technical solution, the beneficial effects of the present invention is：One kind that the present invention is provided is used for virtual reality The multi information body feeling interaction glove system of system and method, by multi-sensor information fusion technology, the hand three to operator Dimension athletic posture and power are accurately recognized and are calculated, is all greatly improved, not only may be used on accuracy of identification and perception range To recognize hand 3 d pose, accurate finger strength estimation can also be carried out；Using the implicit interactions based on sEMG, it is virtual existing Real system provides hand interactive information input mode the most natural, with light glove as virtual reality input equipment, Can reduce tradition operated based on handle or action bars input equipment and reality environment mismatch the operation for causing it is unnatural, The shortcoming for needing short-term regulation to learn；Using two-way interactive mode, body-sensing glove except perceiving hand exercise information as input, Its multi-modal vibrational feedback can simulate touch feedback output, realize the friendship of a kind of people and virtual reality system two-way exchange feedback Mutually mode, can more really simulate the operation by human hand and tactile in virtual reality, bring the operating experience of more immersion.

Description of the drawings

Structural representations of the Fig. 1 for virtual reality interactive system；

Fig. 2 is the reverse side of the multi information body feeling interaction glove system for virtual reality system provided in an embodiment of the present invention Structural representation；

Fig. 3 is the front of the multi information body feeling interaction glove system for virtual reality system provided in an embodiment of the present invention Structural representation；

Fig. 4 is the characteristic schematic diagram of flexible bending sensor provided in an embodiment of the present invention；

Fig. 5 is the external circuitry schematic diagram of flexible bending sensor provided in an embodiment of the present invention；

Fig. 6 is the characteristic schematic diagram of contact pressure sensor FSR provided in an embodiment of the present invention；

Fig. 7 is processor module provided in an embodiment of the present invention and its external circuit structural representation；

Fig. 8 is the circuit of the multi information body feeling interaction glove system for virtual reality system provided in an embodiment of the present invention Connection diagram；

Fig. 9 is the computational methods schematic diagram of hand exercise information provided in an embodiment of the present invention and pose；

Figure 10 is the contrast curve of pretreated sEMG signals provided in an embodiment of the present invention and original electromyographic signal；

Figure 11 is muscle contraction kinetic model schematic diagram provided in an embodiment of the present invention；

Figure 12 is active contractility provided in an embodiment of the present invention and muscle length item function and passive elastic force and muscle The real curve of length item function and matched curve schematic diagram.

In figure：1st, glove bulk；101st, finger section；102nd, wrist portion；103rd, oversleeve portion；2nd, bend sensor group；201～ 205th, flexible bending sensor；3rd, myoelectric sensor group；301～308, myoelectric sensor；4th, inertial sensor group；5th, pressure is passed Sensor group；501～505, pressure transducer；6th, signal operation amplifier module；7th, analog-to-digital conversion module；8th, vibrational feedback mould Block；801～804, vibrational feedback unit；9th, processor module；10th, communication module.

Specific embodiment

With reference to the accompanying drawings and examples, the specific embodiment of the present invention is described in further detail.Hereinafter implement Example is for illustrating the present invention, but is not limited to the scope of the present invention.

Virtual reality interactive system is as shown in figure 1, host computer, Binocular displays terminal and body-sensing including virtual reality system Interaction glove, the host computer of virtual reality system refer to the core processor for realizing virtual reality system.

The present embodiment provides a kind of multi information body feeling interaction glove system for virtual reality system, the body feeling interaction handss Set system is the input layer of above-mentioned virtual reality interactive system, for the exercise data by multi-sensor collection arm and hand And transmit to virtual reality system host computer, real motion of the host computer in reality environment middle mold personification hand, and by void In near-ring border, the tactile impressions information of hand is sent to body feeling interaction glove system, and in the form of shaking, feedback user is in virtual environment Sense of touch, body feeling interaction glove system receives the tactile feedback information of host computer, and the vibrational feedback unit for controlling glove system is defeated Go out to shake sense of touch.Virtual reality system host computer can according to the multi-sensor data of body feeling interaction glove system input at Reason and analyze, calculate hand and arm 3 d pose and quantity of motion, including each joint angles and torque, and by virtual reality Picture is transferred to Binocular displays terminal and shows.Body feeling interaction glove system is carried out in the way of bluetooth wirelessly with virtual reality system Communication.

A kind of structure such as Fig. 2 for multi information body feeling interaction glove system for virtual reality system that the present embodiment is provided With shown in Fig. 3, Fig. 2 is glove reverse structure schematic, and Fig. 3 is glove positive structure schematic.The body feeling interaction glove system Including glove bulk 1, sensor assembly, signal operation amplifier module 6, analog-to-digital conversion module 7, vibrational feedback mould/8, process Device module 9 and communication module 10.

Glove bulk 1 is used to carrying and fixing other module groups, is worn on user hand, including finger section 101, wrist Portion 102 and oversleeve portion 103, oversleeve portion 103 can be covered to the fore-arm of user.

Sensor assembly, for gathering motion and the attitude information of hand, including bend sensor group 2, myoelectric sensor Group 3, inertial sensor group 4 and pressure transducer group 5.

Bend sensor group 2 includes 5 flexible bending sensors 201～205, is individually fixed in glove bulk finger section In referring to portion's interlayer on the outside of each fingerstall, for gathering the flexibility of each finger, to obtain finger gesture.It is in the present embodiment, soft Property bend sensor 201～205 adopt flex45 models, its characteristic and external circuitses as shown in Figure 4 and Figure 5, wherein R1 tables Show flex45 flexible bending sensors, wherein, output voltage V_outFor

Myoelectric sensor group 3 is fixed on the forearm oversleeve portion 103 of glove bulk 1, including 8 myoelectric sensors 301～ 308, the obverse and reverse of glove bulk 1 is respectively equipped with four myoelectric sensors, is evenly arranged in forearm oversleeve portion, each myoelectricity Sensor includes contact electromyographic electrode and operational amplification circuit, for gathering the skin that the motion of hand exercise related muscles is produced Surface myoelectric information (sEMG).

Inertial sensor group 4 is fixed on the wrist portion of glove bulk, including an inertial sensor, for gathering at wrist Movable information, in the present embodiment, inertial sensor using ANALOG companies ADIS16460, be the minitype inertial of 14 pins Measuring unit (IMU), built-in 3-dimensional accelerometer and 3-dimensional gyroscope, can detect the kinestate (gravitational acceleration component of hand And angular velocity of satellite motion).

Pressure transducer group 5 is fixed on the inside of each fingerstall of glove bulk finger section, including 5 contact pressure sensings Device 501～505, for measuring finger contact force, the characteristic and external circuitses of contact pressure sensor FSR as shown in fig. 6, its External circuitses are identical with the external circuitses of flex45 flexible bending sensors.

Signal operation amplifier module 6, is fixed on the outside of the oversleeve portion 103 of glove bulk 1, for by myoelectric sensor group The sEMG micro-signals of 3 collections are amplified.

Analog-to-digital conversion module 7, is fixed on the outside in the oversleeve portion 103 of glove bulk 1, for myoelectric sensor group 3 is examined The analogue signal after signal operation amplifier amplifies surveyed is processed and is converted into digital signal, to improve electromyographic signal Sample rate.In the present embodiment, D/A converter module 7 constitutes 8 groups of high accuracy D/A converter modules using ADS1256 chips, should Module connection circuit includes power circuit, peripheral filter circuits and the connected mode with Arm9CPU.

Vibrational feedback module 8, is fixed on the inside of the fingerstall of 1 finger section 101 of glove bulk and locates, including 4 vibrational feedback lists Unit 801～804, for the hand tactile data of virtual reality system is fed back in the way of shaking hand.

Processor module 9, is fixed on the outside of the wrist portion 102 of glove bulk 1, for collecting and processing analog-to-digital conversion module 7 signal, and calculate attitude, displacement, contact force and the contactless force of hand, and the hand tactile for receiving virtual reality system Information simultaneously feeds back to vibrational feedback module 8.In the present embodiment, processor module 9 is using the Samsung based on Arm9 master control cores S3C2440CPU, as shown in Figure 7.

Communication module 10, is fixed on by processor module 9, is bluetooth communication, for connecting processor module 9 and void Intend the host computer of reality system, realize communication for information and communication.

The circuit connection diagram of the multi information body feeling interaction glove system for virtual reality system that the present embodiment is provided is such as Shown in Fig. 8, as the sample frequency to electromyographic signal requires higher (>=200Hz), and the simulation IO of excessive use S3C2440 is difficult To reach sampling precision, therefore 8 channel analog signals of myoelectric sensor are amplified by operational amplifier, and are turned by high-precision AD Mold changing block ADS1256 is converted to digital signal.A/D module data port is connected with Digital I/O area, and is connected with cpu clock area, is received Clock signal.Other sensors data are by simulating I O read.Wherein pressure transducer takes 4 analog input ends of CPU Mouthful, flexible bending sensor takes 5, and vibrational feedback needs to connect 6 simulation input ports, is both connected to the simulation IO of CPU Area.CPU will carry out data transfer with bluetooth module in the way of serial communication, and connectivity port also is located at Digital I/O area, realize with Radio communication between outside host computer.

It is a kind of that hand exercise letter is carried out using the above-mentioned multi information body feeling interaction glove system for virtual reality system The computational methods of breath and pose, as shown in figure 9, the method is loaded into the processor module 9 of above-mentioned glove system in the form of program, In real time hand exercise and attitude information are calculated, while receiving the feedback information of virtual reality system host computer and controlling shake Dynamic feedback module 8 realizes two-way interactive simulating tactile, and concrete grammar is as follows.

Step 1：The signal of each sensor group in real-time collecting sensor assembly, and carry out analog digital conversion.

Step 2：Design digital filter algorithm, is filtered to each digital signal and noise suppression preprocessing.

The pretreatment of sEMG signals, including two links of filtering and noise reduction and all wave rectification, preprocessing process are digital using IIR Band filter carries out denoising, mainly filters high-frequency noise and 50Hz Hz noises.Formula (1) is public for the calculating of preprocessing process Formula.

Wherein, L represents bandpass filter function；E (t) is original sEMG signals；For average in access time window Value；U (t) is pretreated sEMG signals；u₀For side-play amount.

Pretreated sEMG signals are as shown in Figure 10 with the contrast of original electromyographic signal.

SEMG signals are gathered by myoelectric sensor, and the action potential sequence provided by multiple active movement units is fine along flesh Dimension is propagated, and after the volume conductor filtering constituted via fat/skin, time for presenting in skin surface and spatially comprehensive is folded Plus result, with more natural control mode, its be ahead of actual motion generation, action anticipation can be carried out, contain muscle The abundant informations such as power, amount of articulation.Based on these features, estimate that finger strength can reach good real-time estimation by sEMG Effect.

Other signals carry out pretreatment using existing digital filter Denoising Algorithm.

Step 3：Each finger camber is calculated according to the data of bend sensor group collection, according to the number of myoelectric sensor group According to the activity for calculating finger related muscles, wrist linear movement and the anglec of rotation are calculated according to the data of inertial sensor group, According to the data acquisition finger contact force of pressure transducer group.

Flexible sensor direct detection of the finger camber by body-sensing data glove.

The calculating of hand track, i.e., solved by the three-dimensional accelerometer signal of inertial sensor and three-dimensional gyroscope signal Three-dimensional perspective residing for hand.It is P to define rotational steps first, and the rotational steps are the angular velocity absolute value in T sampled point Sum, be shown below：

Wherein, abs () is to ask signed magnitude arithmetic(al),For the axial angular velocity of t-th sample point i.Define hand appearance Two kinds of method for solving of state (three-dimensional perspective), are calculated by accelerometer and gyro data respectively and are obtained：

Method 1：Joint angles are calculated using acceleration transducer：Vector R of three directional acceleration signals is obtained first, Such as following formula：

Wherein, a_x、a_y、a_zRespectively the acceleration signal in three directions, then can obtain the angle of all directions angle_i, as shown in formula (4), wherein, i=x, y, z.

Method 2：Joint angles are calculated using gyroscope, such as formula (5)；

angle_i(n)=angle_i(n-1)+w_i·T (5)

Wherein, angle_iThe joint angles of (n) for current sampling point, angle_i(n-1) be calculate previous sampled point joint Angle value, w_iFor the angular velocity in the i directions of collection, time rate of changes of the T for gyroscope gathered data.

As rotational steps p ＜ p₁When, it is stipulated that for static or low speed, using method 1 is calculated；Work as p₁≤p≤p₂When, it is stipulated that speed Degree is moderate, and the result weighted sum of using method 1 and 2 must be calculated；As p ＞ p₂When, it is stipulated that speed, using method 2 are calculated.

Wherein, p₁、p₂And p₃Respectively 20%, 40% and the 60% of hand exercise most angular velocity.

In the present embodiment, in method 1, acceleration transducer is accurate to the angle calculation of static and lower-speed state, but to speed The situation of degree mutation has significantly deviation, and in method 2, gyroscope is accurate to the angle calculation under rotary motion, but has zero under static state Point skew and slight saltus step.Therefore the present embodiment tries to achieve hand gestures by the way of two methods combine.

Step 4：Many muscle kinetic models are set up, Fingers power is estimated, refer to that power includes contact force and noncontact Power, specifically includes following steps.

Step 4.1：The many muscle kinetic models set up between muscle activation degree and muscular force, for by the correlation being input into Muscle activation degree calculates muscular force, and the muscular force is used for the finger power for estimating finger, and the concrete grammar that model is set up is as follows.

Step 4.1.1：Set up activity model.Activity model can be used for the activation degree for calculating muscle, activate journey Degree is that reaction the important of muscular force size is measured, the direct reaction activation level of muscle.In the present embodiment, muscle activation degree is situated between Between 0 to 1.Muscle activation degree is calculated according to formula (6) by pretreated sEMG signals.

Wherein, a (t) represents the activity of t, and u (t) is the pretreated sEMG signals of t.

Step 4.1.2：Muscle contraction kinetic model is set up, the model abbreviation Hill models, are by a large amount of anatomy The simulation mechanics of muscle model that experiment and medical data analysis are set up, by the macroscopical mechanism model body of muscle fiber change of microcosmic It is existing, reflect the forming process of muscular force.The muscle model structure of Hill models is as shown in figure 11.Including generation active contractilityActive contractile unit (CE) and the passive elastic force of generation connected in parallelFlexible element (PE).In figure specifically related to Parameter listed by table 1.

1 Hill Parameters in Mathematical Model of table

Tendon units power F in Hill models₁ ^mtBy active contractilityWith passive elastic forceSuperposition F^mProduce, can To be described as：

Wherein, φ is the angle of meat fiber and tendon.Active contractilityBy itself and muscle length item functionWith Muscle contraction speed term function f_VThe flesh contractility such as (v), muscle activation degree a (t) and muscle maximumRepresent, passive elastic forceBy itself and muscle length item functionWith the flesh contractility such as muscle maximumRepresent, as shown in formula (8).

It is approximated as follows hypothesis：1) included angle of meat fiber and tendon keeps constant, takes 0.2；2) muscle contraction speed pair The impact of active force change can be ignored, i.e. f_V(v)≈1；3) ratio of rigidity of tendon is larger, i.e. tendon length is held essentially constant, that is, return One changes muscle lengthCan be calculated by following formula：

It is to be unfavorable for modeling due to Hill models excessively complexity to make this hypothesis, and parameter is difficult to obtain, after approximating assumption Can significantly facilitate in line computation on the premise of certain calculation accuracy is kept.

Active contractility and muscle length item functionWith passive elastic force and muscle length item functionLetter Number form formula, by the existing data based on medical research as curve matching setting up.Figure 12 isWithFitting Data in front and back,It is fitted by second order Gauss function,It is fitted by exponential function, it is resulting Fitting function formula is respectively as shown in formula (10) and formula (11).

Step 4.1.3：By calculated muscle activation degree a (t) and normalization muscle lengthFormula (7) is brought into formula (9) muscle force prediction value, is obtained.

Step 4.2：The finger that the finger contact force detected using pressure transducer group is estimated with many muscle kinetic models Refer to that power sets up state equation, the finger power of finger is accurately estimated by EKF, concrete grammar is as follows.

Step 4.2.1：Set up the state equation with Fingers power as state.If Fingers power F^mt(Fingers power is multiple The sum of tendon units power) and the power derivativeFor the state of Kalman filtering, the Fingers power of n-th sampled point and its lead Shown in several state such as formula (12) and formula (13), wherein, w₁N () is the process noise in Fingers power state description equation, w₂ N () is the process noise in finger strength derivative state descriptive equation, T is the sampling time.

The state vector of n-th sampled point is designated as x (n), remembers estimated value F of Fingers power^cAnd its derivativeComposition output VectorFormula (12) and formula (13) simultaneous obtain state equation as follows：

Wherein, z (n) is the observer state for referring to power, is determined by measured value, process noises of the w (n) for model, and v (n) is survey Amount noise, is white Gaussian noise, i.e. w=N (0, Q), v=N (0, R), wherein, Q and R are constant.NoteH= [1 1]。

Step 4.2.2：Calculation error matrix simultaneously updates state equation.Calculated by formula (15) by last sampling point value first and worked as The least mean-square error matrix P (n | n-1) of front sampled point, P (n-1 | n-1) are the least mean-square error square of a upper sampled point Battle array.For first sampled point, initialization P (1 | 1) is unit battle array.

P (n | n-1)=A P (n-1 | n-1) A^T+Q (15)

Then, amendment is updated to current state by least mean-square error matrix, including calculation error gain K (n), is repaiied On the occasion of the least mean-square error matrix P (n | n) of x (n | n) and current state, as shown in formula (16).

Wherein, x (n | n-1) represents the current state value calculated under the conditions of preceding state is.

By the state of formula (16) newer (14) state equation, the finger strength estimated value of each sampled point can be obtained, That is state equation output.

Step 5：The finger camber for obtaining, wrist linear movement and the anglec of rotation, finger contact force and contactless force are believed Breath is sent to communication module port, while the tactile feedback information of virtual reality system host computer is received, it is anti-for controlling vibrations Feedback module.

A kind of multi information body feeling interaction glove system and method for virtual reality system that the present invention is provided, by many Sensor data fusion technology, is accurately recognized and is calculated to the hand three-dimensional motion attitude and power of operator, in identification All it is greatly improved on precision and perception range, hand 3 d pose not only can be recognized, accurate finger strength can also be carried out Estimate；Using the implicit interactions based on sEMG, hand interactive information input side the most natural is provided for virtual reality system Formula, with light glove as virtual reality input equipment, it is possible to reduce tradition is based on handle or action bars input equipment with void Intend actual environment and operate the shortcoming that the operation that mismatch is caused is unnatural, need short-term regulation to learn；Using two-way interactive mode, Except perceiving hand exercise information as input, its multi-modal vibrational feedback can simulate touch feedback output to body-sensing glove, real A kind of existing people and the interactive mode of virtual reality system two-way exchange feedback, can more really simulate the handss in virtual reality Portion operates and tactile, brings the operating experience of more immersion.

Finally it should be noted that：Above example only to illustrate technical scheme, rather than a limitation；Although With reference to the foregoing embodiments the present invention has been described in detail, it will be understood by those within the art that：Which still may be used To modify to the technical scheme described in previous embodiment, or which part or all technical characteristic are equal to Replace；And these modifications or replacement, do not make the essence of appropriate technical solution depart from the model limited by the claims in the present invention Enclose.

Claims (6)

1. a kind of multi information body feeling interaction glove system for virtual reality system, it is characterised in that：The system includes glove Body (1), sensor assembly, signal operation amplifier module (6) and analog-to-digital conversion module (7), vibrational feedback module (8), place Reason device module (9) and communication module (10)；

The glove bulk (1) is worn on user hand for carrying and fix other module groups, including finger section (101), Wrist portion (102) and oversleeve portion (103), oversleeve portion (103) can be covered to the fore-arm of user；

The sensor assembly, for gathering motion and the attitude information of hand, including bend sensor group (2), myoelectricity sensing Device group (3), inertial sensor group (4) and pressure transducer group (5)；The bend sensor group (2) is fixed on glove bulk (1) In referring to portion's interlayer on the outside of each fingerstall of finger section, for gathering digital flexion degree, including 5 flexible bending sensors；It is described Myoelectric sensor group (3) is fixed on the forearm oversleeve portion (103) of glove bulk (1), for gathering hand exercise related muscles fortune The skin surface myoelectric information (sEMG) of movable property life, including 8 myoelectric sensors；The inertial sensor group (4) is fixed on handss The wrist portion (102) of set body (1), for gathering the movable information at wrist, including three-dimensional accelerometer and three-dimensional gyroscope； The pressure transducer group (5) is fixed on the inside of each fingerstall of glove bulk (1) finger section (101), connects for measuring finger Touch, including 5 contact pressure sensors；

The signal operation amplifier module (6), is fixed on the outside of glove bulk (1) forearm oversleeve portion (103), for by myoelectricity The sEMG micro-signals that sensor group (3) is gathered are amplified；

The analog-to-digital conversion module (7), is fixed on the outside of glove bulk (1) forearm oversleeve portion (103), for myoelectricity is sensed The analogue signal after signal operation amplifier amplifies that device group (3) is detected is processed and is converted into digital signal, to carry The sample rate of high electromyographic signal；

The vibrational feedback module (8), is fixed on the inside of the fingerstall of glove bulk (1) finger section (101) and locates, for virtually showing The hand tactile data of real system feeds back to hand, including 4 vibrational feedback units in the way of shaking；

The processor module (9), is fixed on the outside of the wrist portion (102) of glove bulk (1), turns for collecting and processing modulus The signal of mold changing block (7), and attitude, displacement, contact force and the contactless force of hand are calculated, and receive virtual reality system Hand tactile data simultaneously feeds back to vibrational feedback module (8)；

The communication module (10), is fixed on processor module (9) side, for connecting processor module (9) and virtual reality system The host computer of system, realizes communication for information and communication, and the communication module (10) is bluetooth communication module.

2. the multi information body feeling interaction glove system for virtual reality system described in a kind of employing claim 1 carries out hand The computational methods of movable information and pose, it is characterised in that：The method is loaded into the process of above-mentioned glove system in the form of program Device module (9), is calculated to hand exercise and attitude information in real time, while receiving the feedback letter of virtual reality system host computer Cease and control vibrational feedback module (8) to simulate tactile, realize two-way interactive, concrete grammar is as follows：

Step 1：The signal of each sensor group in real-time collecting sensor assembly, and carry out analog digital conversion；

Step 2：Design digital filter algorithm, to analog digital conversion after each digital signal be filtered and noise suppression preprocessing；

Step 3：The data gathered according to bend sensor group (2) calculate each finger camber, according to myoelectric sensor group (3) Data calculate the activity of finger related muscles, according to the three-dimensional accelerometer signal and three-dimensional gyro of inertial sensor group (4) Instrument signal of change wrist linear movement and the three-dimensional perspective of the anglec of rotation, i.e. hand gestures, according to the number of pressure transducer group (5) According to acquisition finger contact force；

Step 4：Many muscle kinetic models are set up, Fingers power is estimated, refer to that power includes contact force and contactless force, had Body is comprised the following steps：

Step 4.1：Carry out the finger power based on many muscle kinetic models to predict, set up many between muscle activation degree and muscular force Muscular motivation model, for calculating muscular force by the related muscles activity being input into, the muscular force is used for predicting Fingers power；

Step 4.2：The Fingers estimated with many muscle kinetic models using the finger contact force that pressure transducer group (5) is detected Power sets up state equation, accurately estimates Fingers power by EKF；

Step 5：The finger camber for obtaining, wrist linear movement and the anglec of rotation, finger contact force and contactless force information are sent out Communication module port is delivered to, while the tactile feedback information for receiving virtual reality system host computer is used to control vibrational feedback module (8)。

3. computational methods of a kind of hand exercise information according to claim 2 and pose, it is characterised in that：The step In 2, denoising is carried out using IIR digital band-pass filters to the preprocessing process of sEMG signals, mainly filter high-frequency noise and 50Hz Hz noises, computing formula of the formula (1) for preprocessing process；

$u\left(t\right)=L\left(\right|e\left(t\right)-\stackrel{\‾}{e}\left(t\right)\left|\right)-u_0---\left(1\right)$

Wherein, L represents bandpass filter function；E (t) is original sEMG signals；For the meansigma methodss in access time window；u T () is pretreated sEMG signals；u₀For side-play amount.

4. computational methods of a kind of hand exercise information according to claim 3 and pose, it is characterised in that：The step In 3, the computational methods of wrist linear movement and the anglec of rotation are：

Definition rotational steps are p, and the rotational steps are the sum of the angular velocity absolute value in T sampled point, are shown below：

$p=\underset{t=1}{\overset{T}{\Σ}}abs\left(w_i^t\right)---\left(2\right)$

Wherein, abs () is to ask signed magnitude arithmetic(al),For the axial angular velocity of t-th sample point i；

Two kinds of method for solving of hand gestures (three-dimensional perspective) are defined, is calculated by accelerometer and gyro data respectively and is obtained：

Method 1：Joint angles are calculated using acceleration transducer：

Determine vector R of three directional acceleration signals of x, y, z, be shown below：

$R=\sqrt{a_x^2+a_y^2+a_z^2}---\left(3\right)$

Wherein, a_x、a_y、a_zThe respectively acceleration signal in three directions of x, y, z；

Determine angle angle of all directions_i, as shown in formula (4), wherein, i=x, y, z.

${\mathrm{angle}}_i=\frac{180}{\mathrm{\π}}arccos\left(\frac{a_z}{R}\right)---\left(4\right)$

Method 2：Joint angles are calculated using gyroscope, as shown in formula (5)；

angle_i(n)=angle_i(n-1)+w_i·T₁ (5)

Wherein, angle_iThe joint angles of (n) for current sampling point, angle_i(n-1) be previous sampled point joint angles, w_i For the angular velocity in the i directions of collection, T₁For the time rate of change of gyroscope gathered data；

As rotational steps p ＜ p₁When, it is static or lower-speed state, using method 1 calculates hand three-dimensional perspective；Work as p₁≤p≤p₂ When, speed is moderate, and the result weighted sum of using method 1 and 2 calculates hand three-dimensional perspective；As p ＞ p₂When, speed is used Method 2 calculates hand three-dimensional perspective；Wherein, p₁、p₂And p₃Respectively 20%, 40% and the 60% of hand exercise maximum angular rate.

5. computational methods of a kind of hand exercise information according to claim 4 and pose, it is characterised in that：The step 4.1 carry out the finger power based on many muscle kinetic models predicts, specifically includes following steps：

Step 4.1.1：Activity model is set up, for calculating the activation degree of muscle, the size of muscular force, muscle activation is reacted Degree is calculated according to formula (6) by pretreated sEMG signals；

$a\left(t\right)=\frac{e^{\mathrm{\&lambda;}u\left(t\right)}-1}{e^{\mathrm{\&lambda;}}-1},-3<\mathrm{\&lambda;}<0---\left(6\right)$

Wherein, a (t) represents the activity of t, and u (t) is the pretreated sEMG signals of t；

Step 4.1.2：Muscle contraction kinetic model, abbreviation Hill models are set up, the model includes producing active contractility Active contractile unit (CE) and the passive elastic force of generation connected in parallelFlexible element (PE)；

Tendon units power F in Hill models₁ ^mtBy active contractilityWith passive elastic forceSuperposition F^mProduce, such as following formula It is shown；

$F_1^{mt}=F^m\&CenterDot;cos\left(\mathrm{\&phi;}\right)=(F_A^m+F_P^m)\&CenterDot;cos\left(\mathrm{\&phi;}\right)---\left(7\right)$

Wherein, φ is the angle of meat fiber and tendon；Active contractilityBy itself and muscle length item functionWith muscle Contraction speed item function f_VThe flesh contractility F such as (v), muscle activation degree a (t) and muscle maximum₀ ^mRepresent, passive elastic force By itself and muscle length item functionWith the flesh contractility such as muscle maximumRepresent, as shown in formula (8)；

$\left\{\begin{array}{c}{F}_A^m=f_A\left({\tilde{l}}^m\right)\&CenterDot;f_V\left(v\right)\&CenterDot;a\left(t\right)\&CenterDot;F_0^m\\ F_P^m=f_P\left({\tilde{l}}^m\right)\&CenterDot;F_0^m\end{array}\right.---\left(8\right)$

It is approximated as follows hypothesis：The included angle of meat fiber and tendon keeps constant, takes 0.2；Muscle contraction speed becomes to active force The impact of change can be ignored；The ratio of rigidity of tendon is larger, i.e. tendon length is held essentially constant；

Then normalization muscle lengthCan be calculated by following formula：

${\tilde{l}}^m=\frac{l^m}{l_0^m}---\left(9\right)$

Wherein, l^mRepresent muscle length,Represent optimal muscle length；

Active contractility and muscle length item functionWith passive elastic force and muscle length item functionFunction shape Formula, by the existing data based on medical research as curve matching setting up, resulting fitting function formula is respectively such as formula (10) and shown in formula (11)；

$f_A\left({\tilde{l}}^m\right)=0.9866\&times;e^{-\frac{{({\tilde{l}}^m-1.073)}^2}{{0.2952}^2}}+0.6533\&times;e^{-\frac{{({\tilde{l}}^m-0.7201)}^2}{{0.1759}^2}}---\left(10\right)$

$f_P\left({\tilde{l}}^m\right)=0.000065\&times;e^{6.47{\tilde{l}}^m}---\left(11\right)$

Step 4.1.3：By muscle activation degree a (t) for obtaining and normalization muscle lengthFormula (7) is brought into formula (9), is obtained Muscle force prediction value.

6. computational methods of a kind of hand exercise information according to claim 5 and pose, it is characterised in that：The step In 4.2, the method for the accurate estimation of EKF is：

Step 4.2.1：Set up the state equation with Fingers power as state；

If Fingers power F^mtWith the derivative of the powerFor the state of Kalman filtering, the Fingers power of n-th sampled point and its lead Shown in several state such as formula (12) and formula (13)；

$F^{mt}\left(n\right)=F^{mt}(n-1)+{\stackrel{\&CenterDot;}{F}}^{mt}(n-1)T+w_1\left(n\right)---\left(12\right)$

${\stackrel{\&CenterDot;}{F}}^{mt}\left(n\right)={\stackrel{\&CenterDot;}{F}}^{mt}(n-1)+w_2\left(n\right)---\left(13\right)$

Wherein, w₁N () is the process noise in Fingers power state description equation, w₂N () is finger strength derivative state descriptive equation In process noise, T is the sampling time；

The state vector of n-th sampled point be x (n), estimated value F of Fingers power^cAnd its derivativeConstituting output vector isFormula (12) and formula (13) simultaneous obtain state equation as follows：

$\left\{\begin{array}{c}x\left(n\right)=\left[\begin{array}{cc}1& T\\ 0& 1\end{array}\right]x(n-1)+w\left(n\right)\\ y\left(n\right)=\left[\begin{array}{cc}1& 1\end{array}\right]z\left(n\right)+v\left(n\right)\end{array}\right.---\left(14\right)$

Wherein, z (n) is the observer state for referring to power, is determined by measured value, process noises of the w (n) for model, and v (n) makes an uproar for measurement Sound, is white Gaussian noise, i.e. w=N (0, Q), v=N (0, R), wherein, Q and R are constant；NoteH=[1 1]；

Step 4.2.2：Calculation error matrix simultaneously updates state equation；

Step 4.2.2.1：The least mean-square error matrix P (n of current sampling point are calculated by formula (15) by the value of a upper sampled point | n-1), P (n-1 | n-1) is the least mean-square error matrix of a upper sampled point；

P (n | n-1)=A P (n-1 | n-1) A^T+Q (15)

For first sampled point, initialization least mean-square error matrix P (1 | 1) is unit battle array；

Step 4.2.2.2：Least mean-square error matrix according to obtaining is updated amendment to current state, such as formula (16) institute Show, least mean-square error matrix P including calculation error gain K (n), correction value x (n | n) and current state (n | n)；

$\left\{\begin{array}{c}K\left(n\right)=P\left(n\right|n-1)\&CenterDot;H^T{\&lsqb;R\left(n\right)+H\&CenterDot;P\left(n\right|n-1)\&CenterDot;H^T\&rsqb;}^{-1}\\ x\left(n\right|n)=A\&CenterDot;x\left(n\right|n-1)+K\left(n\right)\&lsqb;z\left(n\right)-H\&CenterDot;x\left(n\right|n-1)\&rsqb;\\ P\left(n\right|n)=\&lsqb;I-K\left(n\right)\&CenterDot;H\&rsqb;P\left(n\right|n-1)\end{array}\right.---\left(16\right)$

Wherein, x (n | n-1) represents the current state value calculated under the conditions of preceding state is；

Step 4.2.2.3：By the state of formula (16) newer (14) state equation, the finger strength for obtaining each sampled point is estimated The output of value, i.e. state equation.