CN112115813A - Human body electromyographic signal labeling method and device and computing equipment - Google Patents
Human body electromyographic signal labeling method and device and computing equipment Download PDFInfo
- Publication number
- CN112115813A CN112115813A CN202010898778.4A CN202010898778A CN112115813A CN 112115813 A CN112115813 A CN 112115813A CN 202010898778 A CN202010898778 A CN 202010898778A CN 112115813 A CN112115813 A CN 112115813A
- Authority
- CN
- China
- Prior art keywords
- joint
- signal
- electromyographic
- labeling
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000002372 labelling Methods 0.000 title claims abstract description 105
- 238000000034 method Methods 0.000 claims abstract description 66
- 238000012549 training Methods 0.000 claims description 49
- 238000004590 computer program Methods 0.000 claims description 18
- 238000003062 neural network model Methods 0.000 claims description 8
- 230000009471 action Effects 0.000 abstract description 26
- 230000008859 change Effects 0.000 abstract description 10
- 238000012360 testing method Methods 0.000 abstract description 7
- 230000000875 corresponding effect Effects 0.000 description 26
- 239000011159 matrix material Substances 0.000 description 13
- 210000000629 knee joint Anatomy 0.000 description 12
- 210000003205 muscle Anatomy 0.000 description 12
- 230000008569 process Effects 0.000 description 10
- 230000001133 acceleration Effects 0.000 description 9
- 230000006399 behavior Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 8
- 210000002414 leg Anatomy 0.000 description 8
- 230000003183 myoelectrical effect Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000002567 electromyography Methods 0.000 description 6
- 210000001503 joint Anatomy 0.000 description 5
- 210000000544 articulatio talocruralis Anatomy 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 4
- 210000004394 hip joint Anatomy 0.000 description 4
- 210000003141 lower extremity Anatomy 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 210000003314 quadriceps muscle Anatomy 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 210000002683 foot Anatomy 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 210000000689 upper leg Anatomy 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 210000000988 bone and bone Anatomy 0.000 description 1
- 244000309466 calf Species 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000004377 microelectronic Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001429 stepping effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The application provides a human body electromyographic signal labeling method, a human body electromyographic signal labeling device and computing equipment, and relates to the technical field of human body behavior analysis, wherein the method comprises the following steps: the method comprises the steps of obtaining an angle signal and an electromyographic signal of a target joint, determining joint torque of the target joint according to the angle signal, and finally marking the electromyographic signal of the target joint according to the joint torque based on a predetermined advance marking amount. According to the technical scheme provided by the application, because the change of the joint moment is not influenced by individual difference and action difference, the electromyographic signals are marked by the joint moment, so that the accuracy of the marking result can be improved; in addition, repeated testing and manual marking are not needed, and therefore marking efficiency can be improved. In addition, when labeling is carried out, the electromyographic signals of the target joint can be labeled according to the joint moment based on the predetermined advance labeling amount, so that the accuracy of the labeling result can be further improved.
Description
Technical Field
The application relates to a human behavior analysis technology, in particular to a human electromyographic signal labeling method, a human electromyographic signal labeling device and computing equipment, and belongs to the technical field of human behavior analysis signal processing.
Background
With the development of microelectronic technology, the development of human behavior analysis technology is driven. The human behavior analysis technology is widely applied to the fields of medical monitoring, auxiliary health treatment, human-type robots, motion prediction and the like at present.
The human body behavior analysis technology is a technology for converting abstract behavior actions into concrete data by identifying the behavior actions of a human body and analyzing the data, wherein the conversion process from the actions to the data is the key of the technology. The current human behavior analysis technology usually adopts the electromyographic signals as analyzed data, and in order to obtain high-quality electromyographic signals, the electromyographic signals need to be labeled to determine the corresponding actions and starting and stopping points of the electromyographic signals.
The existing electromyographic signal labeling work mainly judges the starting and stopping of each segment of electromyographic signal according to personal experience, but the completion of one action needs a plurality of muscles to participate, and the force applying conditions of different muscles are different greatly, so that the starting and stopping of each segment of electromyographic signal are judged by only depending on the personal experience, the starting and stopping time corresponding to the action can be accurately found by testing for many times, and the problems of low efficiency, time waste and labor waste of the existing data labeling work are caused.
Disclosure of Invention
In view of this, the present application provides a human electromyographic signal labeling method, device and computing device, which are used to improve efficiency of data labeling work.
In order to achieve the above object, in a first aspect, an embodiment of the present application provides a method for labeling a human body electromyographic signal, including:
acquiring an angle signal and an electromyographic signal of a target joint;
determining the joint moment of the target joint according to the angle signal;
and marking the electromyographic signal of the target joint according to the joint moment based on a predetermined advance mark amount, wherein the advance mark amount is a generation time difference between the joint moment and the electromyographic signal.
Optionally, determining the joint moment of the target joint according to the angle signal includes:
and inputting the angle signal into a joint moment prediction model trained in advance to obtain the joint moment corresponding to the target joint.
Optionally, labeling the electromyographic signal of the target joint according to the joint torque based on a predetermined amount of previous labeling includes:
marking the electromyographic signals of the target joint according to the predetermined advance marking amount and the joint torque;
optionally, labeling the electromyographic signal of the target joint according to the joint torque based on a predetermined amount of previous labeling includes:
the joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, the labeling model is obtained by training based on a training sample set, the training sample set comprises a plurality of training samples, each training sample comprises a sample electromyographic signal, a sample joint moment and a labeling result of the sample electromyographic signal, and the labeling result of the sample electromyographic signal is determined according to the sample electromyographic signal, the sample joint moment and a pre-determined pre-labeling quantity.
Optionally, the training method of the annotation model includes:
extracting the signal characteristics of the sample electromyographic signals in each training sample;
and training the neural network model to be trained according to the signal characteristics of the sample electromyographic signals in each training sample, the sample joint moment and the labeling result of the sample electromyographic signals to obtain a labeling model.
Correspondingly, the joint moment and the electromyographic signal are input into a pre-trained labeling model to obtain a labeled electromyographic signal, and the method comprises the following steps:
extracting signal characteristics of the electromyographic signals;
and inputting the joint moment and the signal characteristics of the electromyographic signals into a pre-trained labeling model to obtain labeled electromyographic signals.
Optionally, the advanced mark amount is determined according to the ending time of the angle signal and the electromyographic signal corresponding to the target motion.
In a second aspect, an embodiment of the present application provides a human electromyography signal labeling device, including:
the acquisition module is used for acquiring an angle signal and an electromyographic signal of a target joint;
the determining module is used for determining the joint moment of the target joint according to the angle signal;
and the marking module is used for marking the electromyographic signals of the target joints according to joint moments based on predetermined early marking quantities, and the early marking quantities are the generation time difference between the joint moments and the electromyographic signals.
Optionally, the determining module is specifically configured to:
and inputting the angle signal into a joint moment prediction model trained in advance to obtain the joint moment corresponding to the target joint.
Optionally, the labeling module is specifically configured to:
and marking the electromyographic signals of the target joint according to the predetermined advance marking amount and the joint torque.
Optionally, the labeling module is specifically configured to:
the joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, the labeling model is obtained by training based on a training sample set, the training sample set comprises a plurality of training samples, each training sample comprises a sample electromyographic signal, a sample joint moment and a labeling result of the sample electromyographic signal, and the labeling result of the sample electromyographic signal is determined according to the sample electromyographic signal, the sample joint moment and a pre-determined pre-labeling quantity.
Optionally, the training method of the annotation model includes:
extracting the signal characteristics of the sample electromyographic signals in each training sample;
training a neural network model to be trained according to the signal characteristics of the sample electromyographic signals in each training sample, the sample joint moments and the labeling results of the sample electromyographic signals to obtain a labeling model;
correspondingly, the joint moment and the electromyographic signal are input into a pre-trained labeling model to obtain a labeled electromyographic signal, and the method comprises the following steps:
extracting signal characteristics of the electromyographic signals;
and inputting the joint moment and the signal characteristics of the electromyographic signals into a pre-trained labeling model to obtain labeled electromyographic signals.
Optionally, the advanced mark amount is determined according to the ending time of the angle signal and the electromyographic signal corresponding to the target motion.
In a third aspect, an embodiment of the present application provides an electronic device, including: a memory for storing a computer program and a processor; the processor is configured to perform the method of the first aspect or any of the embodiments of the first aspect when the computer program is invoked.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method according to the first aspect or any embodiment of the first aspect.
According to the method for labeling the human body electromyographic signals, the dynamic signals and the electromyographic signals of the target joint can be obtained, the joint moment of the target joint is determined according to the dynamic signals, and finally the electromyographic signals of the target joint are labeled according to the joint moment. Because the change of the joint moment is not influenced by individual difference and action difference, the electromyographic signals are marked by the joint moment, so that the accuracy of the marking result can be improved; moreover, any tested person can be selected, and the tested person can be allowed to do any action for any time, so that the application range is wide; in addition, repeated testing and manual marking are not needed, and therefore marking efficiency can be improved. In addition, when labeling is carried out, the electromyographic signals of the target joint can be labeled according to the joint moment based on the predetermined advance labeling amount, so that the accuracy of the labeling result can be further improved.
Drawings
Fig. 1 is a schematic flow chart of a method for labeling a human electromyographic signal according to an embodiment of the present application;
FIG. 2 is a schematic flow chart illustrating a method for determining joint moments according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram illustrating an electromyographic signal labeling provided in an embodiment of the present application;
FIG. 4 is a comparison of the quantities labeled in advance provided in the examples of the present application;
fig. 5 is a schematic structural diagram of a human electromyographic signal labeling device provided in an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Currently, the labeling of the electromyographic signals is mainly to judge the starting and stopping of each segment of the electromyographic signals according to personal experience, but the completion of one action needs a plurality of muscles to participate, the force application conditions of different muscles are different greatly, and the time when the muscles at different parts generate the electromyographic signals is different, so that the starting and stopping time corresponding to the action can be accurately found by performing multiple tests. On the other hand, because different testees have different habits during actions, the action condition of muscles is also influenced, so in order to accurately label the electromyographic signals, only one tester is often used, the actions during collection are standard actions, and in order to reduce the difference between the actions, the testees cannot act for a long time, so that the limitation is large.
In order to solve the above problems, embodiments of the present application provide a method for labeling a human body electromyographic signal, and a technical solution of the present application is described in detail with specific embodiments below. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
The method for labeling the human body electromyographic signals provided by the embodiment of the application can be applied to electronic equipment such as a notebook computer, a tablet computer or a mobile terminal, and the embodiment of the application does not limit the specific type of the electronic equipment. The electromyographic signals referred to in the embodiments of the present application are all surface electromyographic signals.
Fig. 1 is a schematic flow chart of a method for labeling a human electromyographic signal according to an embodiment of the present application, and as shown in fig. 1, the method includes the following steps:
and S110, acquiring a dynamic signal and an electromyographic signal of the target joint.
The electronic device may be obtained from various sensors placed on the tested person, or the user may also input the dynamic signal and the electromyographic signal of the target joint into the electronic device after the dynamic signal and the electromyographic signal of the target joint are collected by the sensors. The dynamic signal may include an angle signal and may also include a plantar pressure signal. The target joints may include hip joints, knee joints, ankle joints, and the like of both legs of the subject.
Specifically, before acquiring the dynamic signal and the electromyographic signal of the target joint, various sensors for testing may be placed on the body of the subject, and may include an IMU sensor for acquiring an angle signal, a pressure sensor for acquiring a plantar pressure signal, and an electromyographic sensor for acquiring an electromyographic signal. The IMU sensor can comprise a three-axis gyroscope, a three-axis accelerometer and a three-axis magnetometer, the gyroscope can measure angular velocity (namely the rotating speed of the object), and the IMU sensor can multiply the speed and time to obtain the rotating angle of the object in a certain time period; the accelerometer may measure acceleration of the object; the magnetometer can measure the yaw angle in the horizontal direction. The IMU sensor or the electronic device may fuse the three portions of data (collectively referred to as IMU signals) by means of complementary filtering or kalman filtering, and obtain angle signals in three directions of the target joint, that is, the angle signals may be determined based on the IMU signals.
For example, the electromyographic sensor may be fixed on the surface of the rectus femoris, vastus lateralis, biceps femoris, gastrocnemius and tibialis anterior of the lower limb of the subject to obtain the electromyographic signals of the muscles of the lower limb; or the IMU sensor can be placed on the thigh, the lower leg and the foot of the testee to obtain the lower limb hip joint movement angle signal, the knee joint movement angle signal and the ankle joint movement angle signal of the testee; the pressure sensor can also be placed on the sole of the tested person to acquire the sole pressure information of the tested person. After the myoelectric sensor, the pressure sensor and the IMU sensor are placed on the lower limb part of the tested person as required, corresponding actions such as walking, running, jumping and the like can be carried out, and at the moment, the electronic equipment can obtain a large number of myoelectric signals of rectus femoris, vastus femoris, biceps femoris, gastrocnemius and tibialis anterior muscle, angle signals of thigh, calf and foot, and plantar pressure signals.
And S120, determining the joint moment of the target joint according to the dynamic signal.
The electronic device may determine the joint moment of the target joint according to the dynamic signal, and may specifically determine the joint moment of the target joint by using the following methods:
first, the joint moment of the target joint is determined based on an inverse kinematics model.
Fig. 2 is a schematic flowchart of a method for determining joint torque according to an embodiment of the present application, and as shown in fig. 2, the method includes the following steps:
and S121, determining the joint angle of the target joint according to the predetermined attitude rotation matrix and the angle signal.
As mentioned above, the angle signal is acquired by the sensor, and the signal is the motion angle of the target joint in the inertial coordinate system; the joint angle of the target joint is a motion angle of the target joint in a joint coordinate system, before the joint angle of the target joint is determined, the joint coordinate system can be established in advance by taking the target joint as a reference, and the attitude rotation matrix is determined according to the relative position relationship between the IMU sensor corresponding to the angle signal and the target joint. The attitude rotation matrix is a conversion matrix between an inertial coordinate system and a joint coordinate system.
For example, after placing the various sensors, the tester may use the hip joint, the knee joint, and the ankle joint as references to respectively establish a joint coordinate system of the hip joint, a joint coordinate system of the knee joint, and a joint coordinate system of the ankle joint, and determine the posture rotation matrix corresponding to each target joint according to the relative position relationship between the sensor and each target joint.
After the preparation work is completed, the electronic device may determine the joint angle of the target joint according to the determined attitude rotation matrix and the angle signal.
And S122, determining the joint angular velocity and the joint angular acceleration of the target joint according to the joint angle.
The electronic equipment can respectively carry out first-order algebraic difference operation and second-order algebraic difference operation on the joint angle of the target joint to obtain the joint angular velocity and the joint angular acceleration of the target joint.
And S123, determining the joint moment of the target joint based on the inverse dynamics model according to the sole pressure signal, the joint angle, the joint angular velocity and the joint angular acceleration.
The joints of the human body can be regarded as rigid bodies, and the bones can be regarded as rod pieces, so that the lower limbs of the tested person can be regarded as a multi-link structure. Therefore, the electronic device may use the lagrangian method or the newton euler method to establish the inverse dynamics model. In the embodiment of the present application, a newton euler method is taken as an example to establish an inverse dynamics model.
The inverse dynamics model can be expressed by the following formula:
wherein q represents a joint angle of the target joint,
represents the joint angular velocity of the target joint,
represents the joint angular acceleration of the target joint, A (q) represents the inertia matrix of the target joint,
and (3) representing an offset moment comprising a centripetal force and a coriolis force, an external force and an external moment, a muscle moment arm matrix and a gravity moment, F representing a plantar pressure signal, and T representing a joint moment vector of the target joint.
The electronic device can determine the joint moment of the target joint based on the inverse dynamics model according to the sole pressure signal, the joint angle, the joint angular velocity and the joint angular acceleration.
Second, the joint moment of the target joint is determined based on the joint moment prediction model.
In this embodiment, the joint moment prediction model may also be trained in advance according to a training sample of the joint moment prediction model. When determining the joint moment, the electronic device may input the angle signal to a joint moment prediction model trained in advance to obtain the joint moment corresponding to the target joint.
Specifically, the training sample of the joint moment prediction model may include a sample angle signal and a sample joint moment determined according to the sample angle signal, where the sample joint moment may be obtained based on an inverse dynamics model method.
The training process of the joint moment prediction model can comprise the following steps: the electronic equipment can extract the signal characteristics of the sample angle signals in each training sample, and then train the neural network model to be trained according to the signal characteristics of the sample angle signals in each training sample and the sample joint moment to obtain the joint moment prediction model. Wherein, the signal characteristics of the sample angle signals can be extracted by a Fourier transform method or a wavelet transform method; the neural network model may be a deep feedforward neural network, a recurrent neural network, a deep convolutional network, and the like, which is not particularly limited in this embodiment.
It should be noted that, when the joint moment is determined by using the joint moment prediction model trained in advance, the electronic device may not acquire the sole pressure signal of the testee in step S110.
In the embodiment of the application, the joint moment can be directly obtained according to the angle signal by adopting the joint moment prediction model electronic equipment, so that the processing step of the angle signal and the solving process according to an inverse dynamics model are omitted, and the whole electromyographic signal labeling work is more efficient.
And S130, marking the electromyographic signals of the target joint according to the joint moment.
Since the joint moment is a representation of the acting force between the joints, the joint moment changes along with the change of the action when the tested person acts. Therefore, the electromyographic signals of the corresponding target joints can be labeled by joint moments, and the starting and stopping points of the electromyographic signals of the corresponding actions are determined based on the change condition of the joint moments.
For example, fig. 3 is a schematic diagram illustrating an electromyographic signal labeling provided in an embodiment of the present application, and as shown in fig. 3, (a) in fig. 3 is a variation curve of a joint moment of a knee joint, and (b) in fig. 3 is a variation curve of an electromyographic signal of a knee joint. As shown in fig. 3 (b), when the subject changes from the standing posture to the squatting posture, the myoelectric signal generated by the muscle near the knee joint changes; meanwhile, the force applied to the knee joint of the subject when the subject stands is not consistent with the force applied to the knee joint of the subject when the subject squats, and therefore, as shown in fig. 3 (a), the joint moment of the knee joint changes when the subject changes from the standing posture to the squat posture. Namely, the joint moment can reflect the change condition of human body action, and the joint moment and the electromyographic signals have a correlation relationship, and the corresponding electromyographic signals can be labeled by the joint moment. As shown in fig. 3, point a is a starting change point of the joint moment, point B is an ending change point of the joint moment, the electromyographic signal at the corresponding time of point a is point C, and the electromyographic signal at the corresponding time of point B is point D, that is, the electromyographic signal at point C may be labeled by the joint moment of point a, and the electromyographic signal at point D may be labeled by the joint moment of point B, where point a and point B are starting and stopping points of the joint moment corresponding to the above actions, and correspondingly, point C and point D are starting and stopping points of the electromyographic signal corresponding to the above actions.
According to research, the electromyographic signals are generated in the contraction and relaxation process of muscle movement, and the generation of the electromyographic signals is about 30-150 milliseconds ahead of the force of the muscle. In order to improve the accuracy of the labeling result, in the embodiment of the application, the electronic device may label the electromyographic signal of the target joint based on a predetermined advance labeling amount and joint moment. Wherein the advance mark amount is a generation time difference between the joint moment and the electromyographic signal.
Specifically, the joint moment is determined based on the angle signal, so the generation time difference between the joint moment and the myoelectric signal can be determined from the generation time difference between the angle signal and the myoelectric signal. In practical application, if the electronic device acquires an angle signal from the IMU sensor, the generation time difference between the joint moment and the electromyographic signal can be determined according to the generation time difference between the angle signal and the electromyographic signal; if the electronic device acquires the IMU signal from the IMU sensor, the electronic device may determine a generation time difference between the angle signal and the electromyographic signal according to the generation time difference between the IMU signal and the electromyographic signal, and further determine a generation time difference between the joint moment and the electromyographic signal.
Furthermore, there is a time difference between the angle signal and the electromyogram signal in the generation time, and correspondingly there is a corresponding time difference between the angle signal and the electromyogram signal in the end time, so the electronic device can also determine the advance mark amount according to the end time between the angle signal and the electromyogram signal corresponding to the target action.
Specifically, during the test process, the testee may stop for a period of time each time he/she performs an action, and then perform the next action. For example, fig. 4 is a comparison diagram of the advanced labeling amount provided in the embodiment of the present application, and as shown in fig. 4, (a) in fig. 4 is a variation curve of an angle signal of a knee joint, and (b) in fig. 4 is a variation curve of a myoelectric signal of the knee joint. The subject may take a left leg first, then stop taking 1 second, then take a right leg, and then stop taking 1 second. After the action is stopped, the electromyographic signal collected by the electromyographic sensor is ended first, and then the angle signal collected by the IMU sensor is also ended, so the point E is the end point of the angle signal when the tested person steps on the left leg, the point F is the end point of the angle signal when the tested person steps on the right leg, the point G is the end point of the electromyographic signal when the tested person steps on the left leg, and the point H is the end point of the electromyographic signal when the tested person steps on the right leg. Therefore, the end time difference of the electromyographic signal and the angle signal can be determined according to the point E and the point G or according to the point F and the point H, and the labeling advance corresponding to the leg stepping action is further determined. In addition, in order to obtain the labeled lead more accurately, the testee may also repeat the same action for multiple times, and then the average value of the labeled leads is taken as the labeled lead, which is not described herein again.
After the advance mark amount is determined, the electronic equipment can mark the electromyographic signals of the target joint according to the advance mark amount and the joint torque.
Specifically, the electronic device may determine the start and stop points of the electromyographic signal according to the advance mark amount. For example, referring to fig. 3, point a is a starting point of change of joint torque, point B is an ending point of change of joint torque, and the electronic device determines that the start point of the electromyographic signal is point C 'and the end point is point D' when the advance mark amount predetermined by the electronic device is 50 milliseconds.
In order to be more convenient and faster, in this embodiment, a labeling model may be trained first, and then the joint moment and the myoelectric signal are input into the trained labeling model to obtain a labeled myoelectric signal.
Specifically, the labeling model may be obtained by training based on a training sample set, the training sample set may include a plurality of training samples, each training sample may include a sample electromyography signal, a sample joint moment, and a labeling result of the sample electromyography signal, and the labeling result of the sample electromyography signal may be determined according to the sample electromyography signal, the sample joint moment, and a predetermined advance labeling amount.
Wherein, the training process of the label model can comprise the following steps: the electronic equipment can extract the signal characteristics of the sample electromyographic signals in each training sample, and then train the neural network model to be trained according to the signal characteristics of the sample electromyographic signals in each training sample, the sample joint moment and the labeling result of the sample electromyographic signals to obtain a labeling model. Wherein, the signal characteristics of the sample electromyographic signals can be extracted by a Fourier transform method or a wavelet transform method; the neural network model may be a deep feedforward neural network, a recurrent neural network, a deep convolutional network, and the like, which is not particularly limited in this embodiment.
Correspondingly, when the labeling is performed, the electronic device may extract the signal characteristics of the electromyographic signal, and then input the joint moment and the signal characteristics of the electromyographic signal into a pre-trained labeling model to obtain the labeled electromyographic signal. By adopting the labeling model, the electronic equipment can directly obtain the labeled electromyographic signals according to the joint torque and the electromyographic signals, the accuracy of the labeling result is ensured while the advance labeling amount is not required to be obtained, and the whole electromyographic signal labeling work is more efficient.
According to the method for labeling the human body electromyographic signals, the dynamic signals and the electromyographic signals of the target joint can be obtained, the joint moment of the target joint is determined according to the dynamic signals, and finally the electromyographic signals of the target joint are labeled according to the joint moment. Because the change of the joint moment is not influenced by individual difference and action difference, the electromyographic signals are marked by the joint moment, so that the accuracy of the marking result can be improved; moreover, any tested person can be selected, and the tested person can be allowed to do any action for any time, so that the application range is wide; in addition, repeated testing and manual marking are not needed, and therefore marking efficiency can be improved.
In addition, when labeling is carried out, the electromyographic signals of the target joint can be labeled according to the joint moment based on the predetermined advance labeling amount, so that the accuracy of the labeling result can be further improved.
Based on the same inventive concept, as an implementation of the above method, an embodiment of the present application provides a human body electromyographic signal labeling apparatus, where the apparatus embodiment corresponds to the foregoing method embodiment, and for convenience of reading, details in the foregoing method embodiment are not repeated in this apparatus embodiment one by one, but it should be clear that the apparatus in this embodiment can correspondingly implement all the contents in the foregoing method embodiment.
Fig. 5 is a schematic structural diagram of a human electromyography signal labeling device provided in this embodiment of the present application, and as shown in fig. 5, the device provided in this embodiment includes:
the acquisition module 110 is used for acquiring a dynamic signal and an electromyographic signal of a target joint;
a determination module 120, configured to determine a joint moment of the target joint according to the dynamic signal;
and the labeling module 130 is used for labeling the electromyographic signals of the target joint according to the joint moment.
Optionally, the dynamic signal includes an angle signal and a plantar pressure signal, the angle signal is a motion angle of a target joint in an inertial coordinate system, and the determining module 120 is specifically configured to:
determining a joint angle of a target joint according to a predetermined attitude rotation matrix and an angle signal, wherein the attitude rotation matrix is a conversion matrix between an inertial coordinate system and a joint coordinate system, the joint coordinate system is a coordinate system established by taking the target joint as a reference, and the joint angle is a motion angle of the target joint in the joint coordinate system;
determining the joint angular velocity and the joint angular acceleration of the target joint according to the joint angle;
and determining the joint moment of the target joint based on the inverse dynamics model according to the sole pressure signal, the joint angle, the joint angular velocity and the joint angular acceleration.
Optionally, the inverse dynamics model is established based on a newton euler method or a lagrange method.
Optionally, the inverse dynamics model is:
wherein q represents a joint angle of the target joint,
represents the joint angular velocity of the target joint,
represents the joint angular acceleration of the target joint, A (q) represents the inertia matrix of the target joint,
the offset moment is represented, F represents a plantar pressure signal, and T represents a joint moment vector of the target joint.
Optionally, the determining module 120 is specifically configured to:
and determining a posture rotation matrix according to the relative position relation between the sensor corresponding to the angle signal and the target joint.
Optionally, the angle signal is determined based on the inertial measurement unit IMU signal.
Optionally, the labeling module 130 is specifically configured to:
and marking the electromyographic signal of the target joint according to the joint moment based on a predetermined advance mark amount, wherein the advance mark amount is a generation time difference between the joint moment and the electromyographic signal.
Optionally, the determining module 120 is specifically configured to:
and inputting the angle signal into a joint moment prediction model trained in advance to obtain the joint moment corresponding to the target joint.
Optionally, the labeling module 130 is specifically configured to:
and marking the electromyographic signals of the target joint according to the predetermined advance marking amount and the joint torque.
Optionally, the labeling module 130 is specifically configured to:
the joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, the labeling model is obtained by training based on a training sample set, the training sample set comprises a plurality of training samples, each training sample comprises a sample electromyographic signal, a sample joint moment and a labeling result of the sample electromyographic signal, and the labeling result of the sample electromyographic signal is determined according to the sample electromyographic signal, the sample joint moment and a pre-determined pre-labeling quantity.
Optionally, the training method of the annotation model includes:
extracting the signal characteristics of the sample electromyographic signals in each training sample;
training a neural network model to be trained according to the signal characteristics of the sample electromyographic signals in each training sample, the sample joint moments and the labeling results of the sample electromyographic signals to obtain a labeling model;
correspondingly, the joint moment and the electromyographic signal are input into a pre-trained labeling model to obtain a labeled electromyographic signal, and the method comprises the following steps:
extracting signal characteristics of the electromyographic signals;
and inputting the joint moment and the signal characteristics of the electromyographic signals into a pre-trained labeling model to obtain labeled electromyographic signals.
Optionally, the advanced mark amount is determined according to the ending time of the angle signal and the electromyographic signal corresponding to the target motion.
The apparatus provided in this embodiment may perform the above method embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Based on the same inventive concept, the embodiment of the application also provides the electronic equipment. Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 6, the electronic device according to the embodiment includes: a memory 21 and a processor 20, the memory 21 being for storing a computer program 23; the processor 20 is arranged for performing the method according to the above-described method embodiment when invoking the computer program 23.
The electronic device provided by this embodiment may perform the above method embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
Embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the method described in the above method embodiments.
The integrated unit may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable storage medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signal, telecommunication signal, and software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/device and method may be implemented in other ways. For example, the above-described apparatus/device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A method for labeling human electromyographic signals is characterized by comprising the following steps:
acquiring an angle signal and an electromyographic signal of a target joint;
determining joint torque of the target joint according to the angle signal;
and marking the electromyographic signal of the target joint according to the joint moment based on a predetermined advance mark amount, wherein the advance mark amount is a generation time difference between the joint moment and the electromyographic signal.
2. The method of claim 1, wherein determining the joint moment of the target joint from the angle signal comprises:
and inputting the angle signal into a joint moment prediction model trained in advance to obtain the joint moment corresponding to the target joint.
3. The method according to claim 1, wherein the labeling of the electromyographic signals of the target joint according to the joint moments based on a predetermined amount of advance labeling comprises:
and marking the electromyographic signals of the target joint according to the predetermined advance marking amount and the joint torque.
4. The method according to claim 1, wherein the labeling of the electromyographic signals of the target joint according to the joint moments based on a predetermined amount of advance labeling comprises:
inputting the joint moment and the electromyographic signals into a pre-trained labeling model to obtain labeled electromyographic signals, wherein the labeling model is obtained by training based on a training sample set, the training sample set comprises a plurality of training samples, each training sample comprises a sample electromyographic signal, a sample joint moment and a labeling result of the sample electromyographic signal, and the labeling result of the sample electromyographic signal is determined according to the sample electromyographic signal, the sample joint moment and a pre-determined pre-labeling quantity.
5. The method of claim 4, wherein the method for training the label model comprises:
extracting the signal characteristics of the sample electromyographic signals in each training sample;
training a neural network model to be trained according to the signal characteristics of the sample electromyographic signals in each training sample, the sample joint moments and the labeling results of the sample electromyographic signals to obtain a labeling model;
correspondingly, the inputting the joint moment and the electromyographic signal into a pre-trained labeling model to obtain a labeled electromyographic signal includes:
extracting signal characteristics of the electromyographic signals;
and inputting the joint moment and the signal characteristics of the electromyographic signals into the pre-trained labeling model to obtain the labeled electromyographic signals.
6. The method according to any one of claims 1 to 5, wherein the advanced amount of marking is determined according to the ending time of the angle signal and the electromyogram signal corresponding to the target motion.
7. A human electromyographic signal labeling device, comprising:
the acquisition module is used for acquiring an angle signal and an electromyographic signal of a target joint;
the determining module is used for determining the joint moment of the target joint according to the angle signal;
and the marking module is used for marking the electromyographic signals of the target joint according to the joint torque based on a predetermined advance marking amount, wherein the advance marking amount is a generation time difference between the joint torque and the electromyographic signals.
8. An electronic device, comprising: a memory for storing a computer program and a processor; the processor is adapted to perform the method of any of claims 1-6 when the computer program is invoked.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-6.
10. A chip system, comprising a processor coupled with a memory, the processor executing a computer program stored in the memory to implement the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010898778.4A CN112115813B (en) | 2020-08-31 | 2020-08-31 | Labeling method and device for human body electromyographic signals and computing equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010898778.4A CN112115813B (en) | 2020-08-31 | 2020-08-31 | Labeling method and device for human body electromyographic signals and computing equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112115813A true CN112115813A (en) | 2020-12-22 |
| CN112115813B CN112115813B (en) | 2024-08-13 |
Family
ID=73805644
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010898778.4A Active CN112115813B (en) | 2020-08-31 | 2020-08-31 | Labeling method and device for human body electromyographic signals and computing equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112115813B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114343649A (en) * | 2022-01-07 | 2022-04-15 | 中山大学附属第一医院 | Joint isometric contraction muscle force analysis method, equipment and computer readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080221487A1 (en) * | 2007-03-07 | 2008-09-11 | Motek Bv | Method for real time interactive visualization of muscle forces and joint torques in the human body |
| CN106388819A (en) * | 2016-10-28 | 2017-02-15 | 许昌学院 | Human upper limb muscle state monitoring system based on surface electromyogram signals and judging method thereof |
| CN109276245A (en) * | 2018-11-01 | 2019-01-29 | 重庆中科云丛科技有限公司 | A kind of surface electromyogram signal characteristic processing and joint angles prediction technique and system |
| CN109559826A (en) * | 2019-01-30 | 2019-04-02 | 福州大学 | A kind of knee joint torque estimation method based on surface myoelectric and motor message |
| CN110337269A (en) * | 2016-07-25 | 2019-10-15 | 开创拉布斯公司 | Method and apparatus for inferring user intent based on neuromuscular signals |
| CN110522457A (en) * | 2019-09-30 | 2019-12-03 | 南京伟思医疗科技股份有限公司 | A kind of real-time gait analysis method based on current of electric, sensor feedback |
| CN110555472A (en) * | 2019-08-27 | 2019-12-10 | 安徽心之声医疗科技有限公司 | electrocardiosignal data labeling method based on active learning |
-
2020
- 2020-08-31 CN CN202010898778.4A patent/CN112115813B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080221487A1 (en) * | 2007-03-07 | 2008-09-11 | Motek Bv | Method for real time interactive visualization of muscle forces and joint torques in the human body |
| CN110337269A (en) * | 2016-07-25 | 2019-10-15 | 开创拉布斯公司 | Method and apparatus for inferring user intent based on neuromuscular signals |
| CN106388819A (en) * | 2016-10-28 | 2017-02-15 | 许昌学院 | Human upper limb muscle state monitoring system based on surface electromyogram signals and judging method thereof |
| CN109276245A (en) * | 2018-11-01 | 2019-01-29 | 重庆中科云丛科技有限公司 | A kind of surface electromyogram signal characteristic processing and joint angles prediction technique and system |
| CN109559826A (en) * | 2019-01-30 | 2019-04-02 | 福州大学 | A kind of knee joint torque estimation method based on surface myoelectric and motor message |
| CN110555472A (en) * | 2019-08-27 | 2019-12-10 | 安徽心之声医疗科技有限公司 | electrocardiosignal data labeling method based on active learning |
| CN110522457A (en) * | 2019-09-30 | 2019-12-03 | 南京伟思医疗科技股份有限公司 | A kind of real-time gait analysis method based on current of electric, sensor feedback |
Non-Patent Citations (1)
| Title |
|---|
| XU, LIN；RABOTTI, CHIARA；MISCHI, MASSIMO: "Dedicated Adaptive Filtering for Electromyographic Investigation of Neuromuscular Activation", 《 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING》, 5 May 2013 (2013-05-05) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114343649A (en) * | 2022-01-07 | 2022-04-15 | 中山大学附属第一医院 | Joint isometric contraction muscle force analysis method, equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112115813B (en) | 2024-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Lower-limb joint torque prediction using LSTM neural networks and transfer learning | |
| US9078585B2 (en) | Muscle synergy analysis method, muscle synergy analyzer, and muscle synergy interface | |
| Wixted et al. | Validation of trunk mounted inertial sensors for analysing running biomechanics under field conditions, using synchronously collected foot contact data | |
| CN108836344B (en) | Step length step frequency estimation method and device and gait detector | |
| Varela et al. | A kinematic characterization of human walking by using CaTraSys | |
| CN112115813B (en) | Labeling method and device for human body electromyographic signals and computing equipment | |
| CN113515967B (en) | Motion intention recognition model generation method, device, equipment and storage medium | |
| Loose et al. | A Public Dataset of Overground and Treadmill Walking in Healthy Individuals Captured by Wearable IMU and sEMG Sensors. | |
| CN112115812A (en) | Human motion signal labeling method and device and computing equipment | |
| LeMoyne et al. | Implementation of an iPod application as a wearable and wireless accelerometer system for identifying quantified disparity of hemiplegic gait | |
| JP6259301B2 (en) | Moving motion analysis apparatus, method and system, and program | |
| JP6378492B2 (en) | Moving motion analysis apparatus and program | |
| CN112107290A (en) | Systems, methods, and software applications for KAM predicting multiple gait cycles of a subject | |
| CN116458875A (en) | Method for evaluating cognitive condition based on intelligent mobile phone sensor gait analysis | |
| Leite et al. | Gait rehabilitation monitor | |
| Dalton et al. | Substantiating appropriate motion capture techniques for the assessment of nordic walking gait and posture in older adults | |
| Muhamad et al. | Design and Implementation of Wearable IMU Sensor System for Heel-Strike and Toe-Off Gait Parameter Measurement | |
| JP6486200B2 (en) | Mobile motion analysis apparatus, system, and program | |
| Shanthini et al. | Interpretation of Gait Supervising Mechanism Using Sensor Integrated Makeshift and Analysing Pattern by K-Means Clustering Algorithm | |
| Tang | Comparison Marker-Based and Markerless Motion Capture Systems in Gait Biomechanics During Running | |
| Stetter | Wearable Sensors and Machine Learning based Human Movement Analysis–Applications in Sports and Medicine | |
| JP6378491B2 (en) | Moving motion analysis apparatus, method and system, and program | |
| Herasymenko et al. | The Algorithm for Automatic Determination of Human Step Phases | |
| Wasser | Performing gait phase detection on healthy and post-stroke subjects using joint angles derived from IMU sensor data | |
| Nilsson | Measuring Athletic Performance UsingAccelerometers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |