JP5924724B2 - Mouth-mouth movement state estimation method and jaw-mouth movement state estimation device - Google Patents

Description

  The present invention relates to a method for estimating a stomatognathic motion state and a device for estimating a stomatognathic motion state that estimate the state of stomatognathic motion such as tongue movement, swallowing motion, and mastication motion.

  In general, persons with severe disabilities due to spinal cord injury, stroke, cerebral palsy, muscular dystrophy, etc. often have impaired limb motor functions, making it difficult to communicate. Therefore, in recent years, brain waves, viewpoints, expiration, jaws Research on interfaces that extract the intentions of people with disabilities by using slightly remaining functions such as movement of the tongue and movement of the tongue has been actively conducted. Among these, jaw and mouth movements such as tongue movements are likely to remain relatively functional even with cervical spine injury and the like. In particular, the movement of the tongue is governed by the sublingual nerve, and various movements are possible as it is called a hand in the mouth. It is conceivable that the position of the tongue tip is measured and used.

  Conventionally, as a technique for measuring the position of the tongue tip, a technique in which a pressure sensor or a permanent magnet is installed in the mouth has been developed. For example, a technique for providing a pressure sensor in the mouth is for analysis of dysphagia, mastication disorder, pronunciation disorder, etc., but the technique described in Patent Document 1 (Japanese Patent Laid-Open No. 2006-234) is known. It has been. This is a flexible tongue pressure sensor sheet having a plurality of pressure sensors formed in accordance with the palate shape of the subject. When this tongue pressure sensor sheet is used, it is mounted on the inner surface of the upper jaw of the subject, the tongue pressure is detected by each pressure sensor, and a detection signal is transmitted through the wiring for use in analysis.

JP 2006-234 A

By the way, since the tongue pressure sensor sheet must be worn in the mouth, there are problems in hygiene, obstructing speech and eating, and psychological stress on persons with disabilities when wearing time is prolonged. There is a problem that it tends to cause medical accidents such as accidental ingestion, electric shock, and suffocation.
Therefore, it is only necessary to be able to estimate a voluntary movement simply and non-invasively without installing a sensor in the mouth. Conceivable. However, the movement of the tongue consists of lingual tongue muscles, hyoid hyoid muscles, genioglossus muscles, upper longitudinal tongue muscles, lower longitudinal tongue muscles, transverse tongue muscles, vertical tongue muscles, and digastric muscles and pedicles. It is expressed in the hyoid muscle, the hyoid hyoid muscle, the geniohyoid muscle, the subhyoid muscle, the sternohyoid muscle, the sternothyroid muscle, and the hyoid muscle of the thyroid hyoid muscle. Because of the overlapping, it is difficult to accurately measure the muscle activity of a specific muscle. Even if it can be measured accurately, unlike the skeletal muscles that drive the upper and lower limbs, it is difficult to correlate myoelectric potential with exercise on a one-to-one basis. Furthermore, for measuring myoelectric potential of specific muscles, (1) it takes time to search for the optimal electrode position, and (2) it is impossible to accurately separate shallow, deep, near, and far muscle activities. There are many issues to be addressed.

  The present invention has been made in view of the above points, and even if it is a comprehensive detection of a muscular biological signal, it can be associated with the state of jaw and mouth movement on a one-to-one basis, and is non-invasive. It is an object of the present invention to provide a method for estimating a stomatognathic movement state and a device for estimating a stomatognathic movement state, which are capable of estimating the state of the stomatognathic movement with high accuracy and convenience.

The method for estimating the oral cavity movement state of the present invention for achieving such an object is the method for estimating the oral cavity movement state, in which the state of the oral cavity movement state is estimated from the biological signal in the interface for controlling the external device. A biological signal is detected from a predetermined skin surface of the human body reflecting the movement of the muscle related to the jaw and mouth movement, the state of the jaw and mouth movement is estimated based on the detected biological signal by a neural network, and an external device is The configuration is used as an interface for control .

Here, the stomatognathic movement refers to any movement such as a movement of a specific movement site or a complex movement of the entire stomatognathic cavity, and these dynamic or static movements. For example, tongue movement, jaw movement, tooth meshing movement with jaw movement, occlusal movement, chewing movement, swallowing movement, yawning movement, opening movement, closing movement, lip opening / closing movement, vocalization movement, laryngeal bulge ( Various movements such as the movement of the Throat Buddha) can be mentioned.
Here, the biological signal refers to a myoelectric potential signal, a muscle sound signal, a myocardial signal, or the like.

Thus, for example, when the biosignal is a myoelectric signal, the myoelectric signal is detected from a predetermined skin surface of the human body that reflects the movement of the muscle related to the jaw and mouth movement, so that the identification that directly contributes to the jaw and mouth movement Since the detection of the total myoelectric potential is not performed from the muscle of the musculoskeletal system, detection is extremely simple even without anatomical knowledge of the musculoskeletal system. Therefore, with a small number of electrodes (number of amplifiers), it is possible to measure a biological signal including all the muscle activities involved in the shallow mouth, deep portion, near portion, and far portion jaw and mouth movements.
Moreover, even if such a myoelectric signal is used, the state of the stomatognathic movement is estimated by the neural network, so even if the comprehensive myoelectric potential is detected, this and the condition of the stomatognathic movement are one-to-one. Therefore, the state of the stomatognathic movement can be estimated non-invasively, simply, and with high accuracy.
The same applies to other biological signals.

If necessary, the neural network learns in advance based on the teacher signal related to the jaw-and-mouth movement until the error of the state related to the estimation falls within a predetermined range, and after the learning process ends. It has a function of performing an execution estimation process for outputting a state estimated for a newly detected myoelectric potential signal as an execution estimation state.
Thereby, in the learning process, learning is performed until the state error related to estimation falls within a predetermined range, so that the state of the stomatognathic movement can be estimated with high estimation accuracy even with a small number of times of learning. As a result, in the execution estimation process, it is possible to improve the estimation accuracy of the execution estimation state estimated for the newly detected myoelectric potential signal from the subject.

If necessary, the jaw and mouth movement includes the movement of a specific movement part, and the state of the jaw and mouth movement related to the estimation of the specific movement part is defined by the position and / or force of the specific movement part. It is assumed to be configured.
Specific movement parts include parts that can be moved arbitrarily, such as the tongue, lips, and jaws.
Thereby, since the state of the stomatognathic movement is defined by the position and / or force of the specific movement part, the amount of information to be output can be increased, and the communication function can be improved. Therefore, for example, versatility can be increased as an interface for personal computer input or various machine operations. In particular, if the position and force are estimated at the same time, it is possible to communicate two things at the same time. For example, it is possible to control the direction and speed of an electric wheelchair to travel at the same time, and to perform a touch operation on a mobile phone. Can be increased.

In this case, the state of the stomatognathic movement in the specific movement part is a state of contact with another part, and it is effective to be defined by a contact position and / or a contact force with the other part.
Examples of combinations of the contact states include, for example, when the specific movement part is the tongue, the other part is the inner surface of the upper jaw or lower jaw, gums, palate, teeth, etc., and the specific movement part is the jaw. For example, the contact state between the lower jaw teeth and the upper jaw teeth is selected.
As a result, the state of the stomatognathic movement is defined by the contact position and / or contact force between the specific movement part and other part, so it becomes easy to specify the position and specify the force by setting a reference, Accordingly, the number of commands that can be used as an interface and the estimation accuracy can be improved.

  Further, if necessary, the teacher signal related to the specific exercise part is output based on the actual position of the exercise part and / or the measured force value. Since learning is performed in correspondence with actual measurement values, learning can be facilitated, and estimation accuracy can be improved.

Further, if necessary, the neural network may perform other oral and oral movements that occur in addition to the oral and oral movements to be estimated for the specific moving part, when the estimation is performed based on the position and / or force of the specific moving part. All or a part of the state is estimated at the same time.
When there is other jaw-and-mouth movement that occurs in addition to the jaw-and-mouth movement that should be estimated for a specific movement site, it can be understood that the movement is not a predetermined movement of the specific movement site, so that erroneous estimation can be prevented. For example, when the specific movement site is the tongue, swallowing movements that swallow saliva, opening movements, closing movements, yawning movements, occlusal movements, etc. occur when detecting biological signals related to tongue position and force For example, if it is a swallowing motion, if it can be estimated as a swallowing motion state based on the biosignal related to the swallowing motion, the detection of the biosignal to be detected in the detection of the biosignal of the position of the tongue and the force Separation is possible, and erroneous estimation can be prevented and reliable estimation can be performed.

  Furthermore, if necessary, the teacher signal related to the other oral and oral movement is a processing signal uniquely determined so as to correspond to the position and / or force of the specific movement portion or a combined movement thereof. It is set as the structure which is. Further, it is possible to prevent erroneous detection and perform reliable designation.

Further, if necessary, the biological signal is output by a multipoint element group in which a plurality of biological signal detection elements attached to the skin are arranged, and the neural network is connected between the biological signal detection elements of the multipoint element group. The configuration is such that the processing is performed based on all or a part of the combination of the differences between the biological signals. A combination of all or a part of the differences of the biological signals may be performed outside the neural network, or may be performed within the neural network. It is desirable to arrange three or more biological signal detection elements.
For example, when a myoelectric potential signal is detected as a biological signal, a surface electrode is used as the biological signal detection element, and the multipoint element group is configured by an array electrode. As a result, the array electrode is configured using n surface electrodes, and the myoelectric potential at each electrode position is unipolarly induced. Then, select two signals of n, myoelectric potential signals of _n C ₂ = m as is obtained by calculating the difference. It is also possible to select two signals from the m signals and calculate the difference between them. In this method, if there are n electrodes and an nCH amplifier, it can be handled, and the device can be simplified. Furthermore, by arranging electrodes so as to cover the entire muscle group and measuring myoelectric potentials at different interelectrode distances, all muscle activity related to shallow and deep jaw movements can be obtained. Thus, the detection accuracy can be improved.

Then, if necessary, the specific movement site for performing the jaw-and-mouth movement is the tongue, and the state of the jaw-and-mouth movement according to the estimation is defined by the contact position and / or the contact force of the tongue with the upper jaw inner surface. ,
A predetermined skin surface of the human body that reflects the movement of muscles related to the above jaw and mouth movement is set in the back of the mandible between the tip of the lower jaw and the laryngeal protuberance (throat Buddha), and a plurality of biological signals are provided in the back of the jaw A detection element is attached so that a biological signal is detected by the biological signal detection element,
In the above learning process,
A plurality of pressure sensors are arranged at predetermined positions on the inner surface of the upper jaw facing the tongue,
When the neural network makes the tip of the tongue contact each pressure sensor, the detection signal from each pressure sensor is used as a teacher signal,
The contact position and / or contact force with respect to the inner surface of the upper jaw of the tongue is estimated on the basis of the biological signal detected by the biological signal detection element by placing the tongue tip on each pressure sensor, and the contact position and / or Or, based on the teacher signal from the corresponding pressure sensor, until the contact force error is within a predetermined range,
In the above execution estimation process,
The neural network is configured to output the contact position and / or contact force of the tongue tip as an execution estimation state when the tongue tip is brought into contact with the inner surface of the upper jaw.

  In this case, the neural network performs swallowing movement, opening movement, closing movement, yawning movement, which occurs in addition to the jaw-and-mouth movement to be estimated when the estimation is performed based on the contact position and / or contact force of the tongue. It is effective to simultaneously estimate at least one state of the occlusal movement. For example, in the case of a swallowing exercise that swallows saliva, it can be recognized as a “swallowing exercise” and can be separated from a voluntary exercise.

In addition, the device for estimating the stomatognathic movement state of the present invention for achieving the above object is the estimation device for the stomatognathic motion state for estimating the stomatognathic motion state. Biological signal detection means for detecting a biological signal from a predetermined skin surface of the human body to be reflected, and a neural network for estimating and outputting the state of the jaw and mouth movement based on the biological signal detected by the biological signal detection means ,
The neural network includes a learning process that learns in advance based on a teacher signal related to the jaw-and-mouth movement until a state error related to the estimation falls within a predetermined range, and a biological body that is newly detected after the learning process ends. And an execution estimation process for outputting the state estimated for the signal as an execution estimation state.

Thus, for example, when the biosignal is a myoelectric signal, the biosignal detection means detects the myoelectric signal from the predetermined skin surface of the human body reflecting the movement of the muscle related to the jaw and mouth movement. Since comprehensive myoelectric potential detection is performed not from specific muscles that directly contribute to exercise, detection is extremely simple even without anatomical knowledge of the musculoskeletal system. Therefore, with a small number of electrodes (number of amplifiers), it is possible to measure a biological signal including all the muscle activities involved in the shallow mouth, deep portion, near portion, and far portion jaw and mouth movements.
Even if such a myoelectric signal is used, the state of the stomatognathic movement is estimated by a neural network. Therefore, the state of the stomatognathic movement can be estimated non-invasively, easily, and with high accuracy.
Further, in the learning process, learning is performed until the state error related to estimation falls within a predetermined range, so that the state of the stomatognathic movement can be estimated with high estimation accuracy even with a small number of learning times. As a result, in the execution estimation process, it is possible to improve the estimation accuracy of the execution estimation state estimated for the newly detected myoelectric potential signal from the subject.
The same applies to other biological signals.

Further, if necessary, the jaw and mouth movement includes a movement of a specific movement part, and the state of the jaw and mouth movement related to the estimation of the specific movement part is defined by the position and / or force of the specific movement part. It is assumed to be configured.
Thereby, since the state of the stomatognathic movement is defined by the position and / or force of the specific movement part, the amount of information to be output can be increased, and the communication function can be improved. Therefore, for example, versatility can be increased as an interface for personal computer input or various machine operations. In particular, if the position and force are estimated at the same time, it is possible to communicate two things at the same time. For example, it is possible to control the direction and speed of an electric wheelchair to travel at the same time, and to perform a touch operation on a mobile phone. Can be increased.

In this case, the state of the stomatognathic movement in the specific movement part is a state of contact with another part, and it is effective to be defined by a contact position and / or a contact force with the other part.
As a result, the state of the stomatognathic movement is defined by the contact position and / or contact force between the specific movement part and other part, so it becomes easy to specify the position and specify the force by setting a reference, Accordingly, the number of commands that can be used as an interface and the estimation accuracy can be improved.

Further, if necessary, the neural network may perform other oral and oral movements that occur in addition to the oral and oral movements to be estimated for the specific moving part, when the estimation is performed based on the position and / or force of the specific moving part. All or a part of the state is estimated at the same time.
When there is other jaw-and-mouth movement that occurs in addition to the jaw-and-mouth movement that should be estimated for a specific movement site, it can be understood that the movement is not a predetermined movement of the specific movement site, so that erroneous estimation can be prevented. For example, when the specific movement site is the tongue, swallowing movement, opening movement, closing movement, yawning movement, and occlusal movement that swallows saliva occurred when detecting the biological signal related to the position and force of the tongue. In this case, for example, in the case of swallowing movement, if the swallowing movement state can be estimated based on the biological signal related to the swallowing movement, the separation of the biological signal to be detected is detected in the detection of the biological signal of the position of the tongue and the force. Thus, it is possible to prevent erroneous detection and perform reliable estimation.

Furthermore, if necessary, the biological signal detection means includes a multipoint element group in which a plurality of biological signal detection elements attached to the skin are arranged, and the neural network is provided between the biological signal detection elements of the multipoint element group. The processing is performed based on all or a part of the combination of the differences in the biological signals. A combination of all or a part of the differences of the biological signals may be performed outside the neural network, or may be performed within the neural network.
For example, when a myoelectric potential signal is detected as a biological signal, a surface electrode is used as the biological signal detection element, and the multipoint element group is configured by an array electrode. As a result, the array electrode is configured using n surface electrodes, and the myoelectric potential at each electrode position is unipolarly induced. Then, select two signals of n, myoelectric potential signals of _n C ₂ = m as is obtained by calculating the difference. It is also possible to select two signals from the m signals and calculate the difference between them. In this method, if there are n electrodes and an nCH amplifier, it can be handled, and the device can be simplified. Furthermore, by arranging electrodes so as to cover the entire muscle group and measuring myoelectric potentials at different interelectrode distances, all muscle activity related to shallow and deep jaw movements can be obtained. Thus, the detection accuracy can be improved.

  Further, if necessary, it is configured to include a motion state identifying means for identifying the corresponding stomatognathic motion state from the execution estimated state output by the execution estimation process function of the neural network. Since the state of motion of the stomatognath corresponding to the execution estimated state is identified by the motion state identifying means, it is possible to automatically give a predetermined command corresponding to the state of the stomatognathic motion to, for example, an external device. It will be possible to respond immediately to various controls.

According to the present invention, since the biological signal is detected from the predetermined skin surface of the human body, which reflects the movement of the muscle related to the stomatognathic movement, the total muscle is not a specific muscle that directly contributes to the stomatognathic movement. Since it becomes the detection of a biological signal, even if there is no anatomical knowledge of the musculoskeletal system, the detection becomes extremely simple. Therefore, with a simple device, it is possible to measure a biological signal including all the muscle activities related to the jaw and mouth movements in the shallow, deep, near and far parts.
In addition, since the state of the stomatognathic movement is estimated by the neural network, it becomes possible to correlate this with the condition of the stomatognathic movement on a one-to-one basis even when detecting a comprehensive muscle biosignal. Thus, the state of the stomatognathic movement can be estimated non-invasively, simply and with high accuracy.

It is a figure which shows the estimation apparatus of the stomatognathic movement state which concerns on embodiment of this invention. The function of the neural network which concerns on embodiment of this invention is shown, (a) is a learning process, (b) is a figure which shows the execution estimation process. It is a figure which shows the structure of the array electrode of a biological signal detection means in the estimation apparatus of a stomatognathic movement state which concerns on embodiment of this invention. It is a figure which shows typically the output form of the myoelectric potential signal of a biosignal detection means in the estimation apparatus of the stomatognathic movement state which concerns on embodiment of this invention. It is a figure which shows the state which mounted | wore the skin surface of the human body with the array electrode of a biological signal detection means in the estimation apparatus of a stomatognathic movement state which concerns on embodiment of this invention. It is a figure which shows the structure of the mouthpiece which acquires a teacher signal in the estimation apparatus of the stomatognathic movement state which concerns on embodiment of this invention. It is a figure which shows the structure of a neural network in the estimation apparatus of the stomatognathic movement state which concerns on embodiment of this invention. It is a flowchart at the time of estimating a stomatognathic movement state with the estimation apparatus of a stomatognathal movement state which concerns on embodiment of this invention. It is a graph which shows the estimation result of the contact position and contact force of a tongue tip concerning Example 1 of this invention. It is a graph which concerns on Example 2 of this invention and shows the estimation result regarding the continuous movement of a tongue tip. It is a figure which concerns on Example 3 of this invention and shows the structure of the mouthpiece which acquires a teacher signal. It is a graph which concerns on Example 3 of this invention, and shows the estimation result of the contact position and contact force of a tongue tip.

  Hereinafter, a method for estimating a stomatognathic movement state and a device for estimating a stomatognathic movement state according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. The method for estimating the stomatognathic movement state according to the embodiment of the present invention is realized by the estimation device for the stomatognathic movement state according to the embodiment of the present invention, and will be described in the description of this estimation device.

  FIG. 1 to FIG. 5 show an apparatus for estimating the oral cavity movement state according to the embodiment. This estimation apparatus S estimates the state of the stomatognathic movement. The stomatognathic motion includes the motion of a specific motion site. In the embodiment, the specific motion site for performing the stomatognathic motion is the tongue T. It is defined by the contact position and contact force with respect to J. In the embodiment, the state of swallowing movement, opening movement, and yawning movement that occurs in addition to the jaw-and-mouth movement to be estimated for the tongue T is simultaneously estimated at the time of estimation by specifying with the contact position and contact force of the tongue tip Ta.

  In detail, the estimation device S according to the embodiment detects a bioelectric signal (EMG) as a biosignal from a predetermined skin surface of a human body, which reflects the movement of the muscle related to the jaw and mouth movement. And a neural network 10 that estimates and outputs the state of the stomatognathic movement based on the myoelectric potential signal detected by the biological signal detection means 1.

  As shown in FIGS. 1 and 5, the skin surface on which the biosignal detection means 1 detects the myoelectric potential signal is set in the chin back area H between the lower jaw tip Ha and the laryngeal protuberance (throat Buddha) Hb. . Then, the biological signal detection means 1 includes an array electrode 2 as a multipoint element group in which a plurality of surface electrodes E as biological signal detection elements attached to the chin area H are arranged as shown in FIGS. 1, the myoelectric potential signal from the array electrode 2 is output to the neural network 10 via the preprocessing unit 3 as shown in FIG. The preprocessing unit 3 includes an amplification amplifier, a rectifier, an A / D converter, and a biosignal feature extractor (not shown). Also. In the neural network 10, processing is performed based on all combinations of differences in the node potential signal E between the surface electrodes E of the array electrode 2.

The array electrode 2 is configured using n surface electrodes E (E1 to En), and induces a myoelectric potential at each electrode position unipolarly. Then, as shown in FIG. 4, _n C ₂ = m myoelectric potential signals can be obtained by selecting two signals from n and measuring the difference. It is also possible to select two signals from the m signals and calculate the difference between them. In the embodiment, nine surface electrodes E (E1 to E9) are prepared, and are attached to one side of the film sheet 4 in a matrix shape and a pyramid shape at intervals of 20 mm as shown in FIG. Each of the nine lead electrodes 5 provided on the side surface is connected. And by attaching this film sheet 4 to the chin back area H as shown in FIG. 5, the surface electrode E is arrange | positioned so that the whole muscle group may be covered. In this case, ₉ C ₂ = 36 EMG signals are obtained, and in general bipolar induction, 72 electrodes and a 36-CH amplifier are required. In this embodiment, 9 electrodes and a 9-CH amplifier are used. If there is enough.

As shown in FIG. 2, the neural network 10 has a learning process (FIG. 2A) that learns in advance based on a teacher signal related to the jaw and mouth movement until an error in the state related to estimation falls within a predetermined range. And a function for performing an execution estimation process (FIG. 2B) for outputting a state estimated for a newly detected myoelectric potential signal after completion of the learning process as an execution estimation state.
In the learning process, the teacher signal is a detection signal from a plurality of pressure sensors 20 (Ch1 to Chn) arranged at predetermined positions on the inner surface J of the upper jaw facing the tongue T as shown in FIGS. As shown in FIG. 6, in the embodiment, seven pressure sensors 20 (Ch1 to Ch7) are prepared and provided on a mouthpiece 21 made of a flexible resin that is mounted in the upper jaw. In the mounted state, the front, middle, and rear on the center line of the upper jaw inner surface J are arranged at two positions on the left and right sides of the symmetrical side with respect to the center line. The pressure sensor 20 detects the contact position of the tongue tip Ta and the magnitude of the contact force (pressing force), and sends a detection signal to the neural network 10. In this case, the teacher signal is output based on the actual measurement value of the contact position and the contact force of the tongue tip Ta, and learning is performed in correspondence with the actual measurement value in the neural network 10, so that learning can be facilitated. The estimation accuracy can be improved.

  The neural network 10 estimates the contact position and the contact force with respect to the maxillary inner surface J of the tongue tip Ta based on the myoelectric potential signal detected by the surface electrode E by positioning the tongue tip Ta in each pressure sensor 20 and, for example, estimates Learning is performed based on the teacher signal from the corresponding pressure sensor 20 until the mean square error of the contact position and the contact force is 0.0001 or less or the number of repetitions of learning reaches 1000 times.

  As shown in FIG. 7, the neural network 10 according to the embodiment is a three-layer neural network including an input layer 11, an intermediate layer 12, and an output layer 13. The output value of 20 is used as a teacher signal, and learning is performed while updating the weight of the interlayer coupling between the input layer 11 and the intermediate layer 12, and between the intermediate layer 12 and the output layer 13, that is, the weight of each neuron is determined by the back propagation method. By learning, the contact position and contact force of the tongue tip Ta in the oral cavity are simultaneously estimated from the myoelectric potential signal measured on the maxillary inner surface J.

  Further, the neural network 10 simultaneously estimates the swallowing movement, opening movement, and yawning movement states other than the jaw-and-mouth movement to be estimated by the tongue T at the time of estimation by specifying the contact position and contact force of the tongue tip Ta. In the learning process, the neural network 10 performs learning related to estimation of the states of swallowing movement, opening movement, and yawning movement in the same manner as described above, and the teacher signal at this time is a uniquely defined processing signal. For example, as a processing signal, a DC5V voltage is applied by a subject's push button in push button buttons corresponding to swallowing movement, opening movement and yawning movement, respectively.

  In addition, as shown in FIG. 1, the estimation device S according to the embodiment includes a motion state identification unit 30 that identifies a corresponding stomatognathic motion state from the execution estimation state output by the execution estimation process function of the neural network 10. It has. In the embodiment, the contact position and contact force of the tongue tip Ta are identified from the estimated value (execution estimation state) from the neural network 10, or are identified as swallowing movement, opening movement, and yawning movement. In the case of movement or yawning movement, the output is retained, and only the contact position of the tongue tip Ta and the estimation result of the contact force are transmitted to the external device 31 via the interface 32. The external device 31 may be any device such as a display device, a printer, a personal computer, and various other control devices. In the case of swallowing movement, opening movement and yawning movement, it is sent to the external device 31 via the interface 32 and output to a display device or a printer, or used or not used by the control device, etc. It may be used as appropriate.

Therefore, when estimating the contact position and contact force of the tongue tip Ta by the estimation device S of the stomatognathic movement state according to the embodiment, as described above, the estimation is performed in advance by the learning process function of the neural network 10. Learning is performed based on the teacher signal related to the stomatognathic movement until the error of the state falls within a predetermined range. Then, as shown in FIG. 5, the subject is provided with the film sheet with the surface electrode E attached to the chin area H, and the tongue T is moved to estimate the contact position and the contact force of the tongue tip Ta. .
Referring to the flowchart shown in FIG. 8, first, the surface electrode E of the array electrode 2 detects a myoelectric potential signal from the chin back area H (S1) and sends it to the neural network 10 (S2).

In this case, the array electrode 2 is configured using n surface electrodes E, and unipolarly induces myoelectric potential at each electrode position, and then selects two signals from the n electrodes and measures the difference between them. Therefore, _n C ₂ = m myoelectric potential signals can be obtained, so that there are n electrodes and nCH amplifiers, so that the device can be simplified. In addition, by arranging electrodes to cover the entire muscle group and measuring myoelectric potential at different interelectrode distances, all muscle activity related to shallow, deep, near, and far jaw movements is obtained. As a result, the detection accuracy can be improved.
In addition, since the myoelectric potential signal is detected from a predetermined skin surface (back jaw area H) of the human body reflecting the movement of the muscle related to the stomatognathic movement, it is not comprehensive from the specific muscle that directly contributes to the stomatognathic movement. Therefore, even if there is no anatomical knowledge of the musculoskeletal system, the detection becomes extremely simple.

Then, estimation is performed in the neural network 10. In this case, even if the myoelectric signal from the chin area H is used as described above, the neural network 10 estimates the state of the jaw and oral movement, so that even if the myoelectric potential is detected comprehensively, It becomes possible to correlate with the state of the oral movement on a one-to-one basis, so that the state of the stomatognathic movement can be estimated non-invasively, easily and with high accuracy.
That is, in the learning process of the neural network 10, learning is performed until the state error related to the estimation falls within a predetermined range, so that the state of the stomatognathic movement can be estimated with high estimation accuracy even with a small number of times of learning. As a result, in the execution estimation process, it is possible to improve the estimation accuracy of the execution estimation state estimated for the newly detected myoelectric potential signal from the subject.

  When the estimation result from the neural network 10 is output, the movement state identification unit 30 identifies the contact position and contact force of the tongue tip Ta, or identifies any of swallowing movement, opening movement, and yawning movement (S3). . When the contact position and the contact force of the tongue tip Ta are identified (S3Y), they are output to the external device 31 via the interface 32 (S4). On the other hand, when the swallowing movement, the opening movement, or the yawning movement is identified (S3N), for example, the reservation is performed, or the swallowing movement, the opening movement, or the yawning movement is output and processed through the interface 32. (S5). In this case, when a myoelectric signal related to the position and force of the tongue tip Ta that is originally necessary is detected, for example, when a swallowing motion that swallows saliva occurs, based on the myoelectric signal related to this swallowing motion Since it can be estimated as the swallowing movement state, the myoelectric potential signal to be detected can be separated in the detection of the myoelectric potential signal of the position of the tongue tip Ta and the force, and erroneous detection can be prevented and reliable estimation can be performed.

  Moreover, since the state of the stomatognathic movement is defined by the contact position and the contact force of the tongue tip Ta, the amount of information to be output can be increased, and the intention transmission function can be improved. Therefore, for example, versatility can be increased as an interface for personal computer input or various machine operations. In particular, if the position and force are estimated at the same time, it is possible to communicate two matters at the same time, such as controlling the direction and speed of the electric wheelchair to travel at the same time, touch operation on the display panel of a mobile phone, etc. And versatility can be increased. Furthermore, since the state of jaw and mouth movement is defined by the contact position and contact force with respect to the maxillary inner surface J of the tongue tip Ta, it becomes easier to specify the position and the force specification by setting a reference, and the estimation accuracy is improved accordingly. To be able to.

Examples of the present invention will be described below.
<Example 1>
The array electrode 2 was prepared by arranging nine surface electrodes E in a pyramid shape at intervals of 20 mm, as described above (FIGS. 3 to 5). The myoelectric potential signal (EMG signal) at each electrode position was derived by monopolar induction using the mandibular angle as a reference potential. For the myoelectric potential measurement, a myoelectric meter (corresponding to the preprocessing unit 3 in FIG. 1) composed of an electrode lead wire, a preamplifier circuit, and a myoelectric amplifier IC (NB6101HS, Nabtesco Corporation) was used. The amplification factor of the myoelectric meter is 1,950 times, the low cutoff frequency is 2.3 Hz, and the high cutoff frequency is 320 Hz.

  In the mouthpiece 21, as shown in FIG. 6, in order to simultaneously measure the pressing force of the tongue tip Ta and the position of the tongue tip Ta at that time, a sheet-type force sensor (thickness of about 0.2 mm) is used as the pressure sensor 20. 7 sheets of FlexiForce (Nitta Corporation) were pasted. The sensing area at each position is 9.5 mm, and the maximum measurement load is 110N.

In the same manner as described above (FIG. 7), a hierarchical NN composed of an input layer, an intermediate layer, and an output layer was used to estimate the position and force of the tongue tip Ta. The EMG signal is added to the input layer (9 neurons), the measured value of the pressure sensor 20, the swallowing movement identification signal, the opening movement identification signal, and the yawning movement identification signal are added to the output layer (9 neurons), and the weight of each neuron is set. By learning by the error back propagation method, the position and force of the tongue tip Ta were simultaneously estimated from the EMG signal measured on the chin. During swallowing movement, opening movement, or yawning movement, DC5V was input as an identification signal by push button switches corresponding to the swallowing movement, opening movement, and yawning movement, respectively. By simultaneously learning swallowing movements, opening movements and yawning movements, we separated them from voluntary movements. The number of neurons in the intermediate layer is the number of combinations between the two electrodes constituting the array electrode 2 so that the muscle activity of the hyoid and hyoid muscles can be considered simultaneously while actively using the crosstalk of the EMG signal ( ₉ C ₂ = 36).

  The test subject was one healthy adult male (22 years old). The experiment was performed with the array electrode 2 and the mouthpiece 21 attached. After pressing the tongue tip Ta against the pressure sensor 20 from Ch1 to Ch7, press the tongue tip Ta against each sensor again with maximum effort, and finally push button switch Swallowing movement, opening movement and yawning movement were performed while operating with the fingers. These operation times were 1 second, and a weakening time of 2 seconds was set between each operation. With the above experimental procedure, a total of three sets of measurement experiments were conducted.

  After learning the network using the first set of experimental data, the position and force of the tongue tip Ta were estimated for the second and third sets of untrained experimental movements. FIG. 9 shows the estimation results for the first set of experimental data. As a result, it was confirmed that the estimated value of the force pressed against each sensor almost coincided with the actual measurement value (true value) and that the swallowing movement, the opening movement and the yawning movement could be accurately discriminated without erroneous recognition.

<Example 2>
The attachment conditions of the array electrode 2 and the attachment conditions of the pressure sensor 20 were the same as those in Example 1 above, and three movement states of the tongue tip Ta, that is, a vertical flick movement, a right and left flick movement, and a circular movement, were estimated. Here, the vertical flicking motion of the tongue tip Ta is an operation of bringing the tongue tip Ta into contact with the palate and tracing it up or down. The left-right flicking motion of the tongue tip Ta is an operation of bringing the tongue tip Ta into contact with the palate and tracing left or right. The circular movement of the tongue tip Ta is an operation of drawing a circle clockwise or counterclockwise by bringing the tongue tip Ta into contact with the palate. These three operations simulate a touch operation on a display panel of a mobile phone.

Similar to the above (FIG. 7), a hierarchical NN composed of an input layer, an intermediate layer, and an output layer was used. EMG signals are added to the input layer (9 neurons), identification signals corresponding to the vertical and horizontal flicking movements, left and right flicking movements, and circular movements of the tongue tip Ta are added to the output layer (9 neurons), and the weight of each neuron is inverted by error. The movement of the tongue tip Ta was estimated from the EMG signal measured on the back of the jaw by learning with the propagation method.
In the vertical flick motion, the left / right flick motion and the circular motion of the tongue tip Ta, a DC5V or DC2.5V identification signal was input by a push button switch corresponding to each of these motions. Specifically, in the vertical flicking motion of the tongue tip Ta, DC5V was input for the upper flick and DC2.5V was input for the lower flick. In the left / right flick motion of the tongue tip Ta, DC5V was input to the left flick and DC2.5V was input to the right flick. In the circular motion of the tongue tip Ta, DC5V was input clockwise and DC2.5V was input counterclockwise.

  As described above, the test subject is one healthy adult male (22 years old). The experiment was performed with the array electrode 2 and the mouthpiece 21 attached, and while operating the push button switch corresponding to each movement with a finger, the upper flick movement, the lower flick movement, the left flick movement, the right flick movement, the clockwise rotation A circular motion and a counterclockwise circular motion were performed. With the above experimental procedure, a total of three sets of measurement experiments were conducted.

  After learning the network using the first set of experimental data, the movement of the tongue tip Ta was estimated for the second and third sets of untrained experimental movements. FIG. 10 shows estimation results for the first set of experimental data. As a result, it was confirmed that the estimated movement identification signal substantially coincides with the identification signal that is the teaching signal, and that the vertical flicking movement, the left-right flicking movement, and the circular movement of the tongue tip Ta can be accurately discriminated without erroneous recognition.

<Example 3>
As shown in FIG. 11, in the mouthpiece 21, three sheet type force sensors (FlexiForce, Nitta Corporation) having a thickness of about 0.2 mm were attached as the pressure sensor 20. Others were the same as in Example 1, and after pressing the tongue tip Ta against the pressure sensor 20 from Ch1 to Ch3, finally, swallowing and yawning were performed while operating the push button switch with a finger. Other than that, measurement experiments were performed in the same experimental procedure as in Example 1. The results are shown in FIG.

  From this result, even if the position of the tongue tip Ta to be estimated and the combination thereof are freely selected, the estimated value of the force pressed against each sensor substantially coincides with the actual measurement value (true value), and swallowing motion without erroneous recognition It was confirmed that yawning movement could be accurately discriminated.

  In the above-described embodiment, the state of the jaw and mouth movement of the tongue T as the specific movement part is defined by the contact position and the contact force with respect to the upper inner surface J of the tongue tip Ta, but is not necessarily limited thereto. Only the contact position or only the contact force may be used, and may be appropriately changed. In addition, the contact position and / or contact force of the tongue tip Ta is not limited to that on the inner surface J of the upper jaw, and for example, any portion where the tongue tip Ta contacts, such as the inside or outside of the cheek meat, teeth, or lips. The site may be selected and may be changed as appropriate. Further, the number of surface electrodes E of the array electrode 2 and the number of pressure sensors 20 of the mouthpiece 21 are not limited to the above, and may be appropriately changed.

  In the above embodiment, the neural network 10 uses the three-layer neural network 10 including the input layer 11, the intermediate layer 12, and the output layer 13. Or other types of neural networks, and may be changed as appropriate.

  In the above embodiment, the case of estimating the state of the tongue T movement and swallowing movement, opening movement, and yawning movement as the jaw-and-mouth movement is not necessarily limited to this. , Jaw movement, tooth meshing movement with jaw movement, occlusal movement, mastication movement, yawning movement, jaw and lip opening / closing movement, vocalization movement, laryngeal uplift (throat bud) movement, etc. The present invention can also be applied to state estimation and can be changed as appropriate.

  In this case, in the above embodiment, the predetermined skin surface of the human body that reflects the movement of the muscle related to the stomatognathic movement is set in the chin area H, but the present invention is not necessarily limited thereto. Any location or area may be set in relation to the muscles that perform the jaw-and-mouth movement, and may be changed as appropriate. Also, the type, number, size, and the like of the electrodes for detecting the myoelectric potential signal are not limited to the above, and may be changed as appropriate.

  INDUSTRIAL APPLICABILITY According to the present invention, it can be applied to a technique for extracting a person's movement intention from a slight residual function in life support for a severely handicapped person such as a person with severe limb paralysis, and can be expected to be applied to life support equipment and rehabilitation. For example, by drawing a character such as “Aiueo ...” with the tip of the tongue, it is possible to directly input the character into a personal computer or to speak using an artificial throat. It can also be used as an interface for various machine operations, such as driving control of an electric wheelchair, a tongue controller for video games, and a touch operation on a display panel of a mobile phone. Moreover, as a matter of course, it can be provided as a useful technique not only for a severely disabled person but also for a healthy person.

S Estimator for stomatognathic movement T Tongue Ta Tongue J Maxillary inner surface H Back of chin area 1 Biological signal detection means 2 Array electrode (multi-point element group)
E (E1 to En) Surface electrode (biological signal detection element)
DESCRIPTION OF SYMBOLS 3 Pre-processing part 4 Film sheet 5 Lead electrode 10 Neural network 11 Input layer 12 Intermediate layer 13 Output layer 20 (Ch1-Chn) Pressure sensor 21 Mouthpiece 30 Motion state identification means 31 External apparatus 32 Interface

Claims (16)

In the estimation method of the stomatognathic movement state that estimates the state of the stomatognathic movement from the biological signal in the interface for controlling the external device,
A biological signal is detected from a predetermined skin surface of the human body reflecting the movement of the muscle related to the jaw and mouth movement, the state of the jaw and mouth movement is estimated based on the detected biological signal by a neural network, and an external device is A method for estimating a stomatognathic movement state, characterized by being used as an interface for control .   The neural network includes a learning process that learns in advance based on a teacher signal related to the stomatognathic movement until an error in a state related to the estimation falls within a predetermined range, and a biological body that is newly detected after the learning process ends. The method of estimating a stomatognathic movement state according to claim 1, further comprising a function of performing an execution estimation process for outputting a state estimated for the signal as an execution estimation state.   The stomatognathic movement includes a motion of a specific motion site, and the state of the stomatognathic motion related to the estimation of the specific motion site is defined by the position and / or force of the specific motion site, The method for estimating the stomatognathic movement state according to claim 2.   4. The state of the stomatognathic movement in the specific movement part is a state of contact with another part, and is defined by a contact position and / or a contact force with the other part. Method for estimating the oral and oral movements of humans.   5. The method for estimating a stomatognathic movement state according to claim 3 or 4, wherein the teacher signal related to the specific movement part is output based on an actual measurement value of the position and / or force of the movement part.   The neural network is configured to estimate all or one of the states of other jaw-and-mouth movements that occur in addition to the jaw-and-mouth movement to be estimated at the time of the estimation based on the position and / or force of the particular movement site. The method for estimating the stomatognathic movement state according to claim 5, wherein the part is also estimated simultaneously.   The teacher signal related to the other stomatognathic movement is a processing signal uniquely determined so as to correspond to the position and / or force of the specific movement site or a combined movement thereof. The method for estimating the stomatognathic movement state according to claim 6.   The biological signal is output by a multipoint element group in which a plurality of biological signal detection elements attached to the skin are arranged, and the neural network calculates all the differences in the biological signals between the biological signal detection elements of the multipoint element group. Alternatively, the method for estimating the stomatognathic movement state according to any one of claims 2 to 7, wherein the processing is performed based on some combination. The specific movement site for performing the oral and oral movement is the tongue, and the state of the oral and oral movement according to the estimation is defined by the contact position and / or the contact force of the tongue with respect to the upper inner surface of the tongue,
A predetermined skin surface of the human body that reflects the movement of muscles related to the above jaw and mouth movement is set in the back of the mandible between the tip of the lower jaw and the laryngeal protuberance (throat Buddha), and a plurality of biological signals are provided in the back of the jaw A detection element is attached so that a biological signal is detected by the biological signal detection element,
In the above learning process,
A plurality of pressure sensors are arranged at predetermined positions on the inner surface of the upper jaw facing the tongue,
When the neural network makes the tip of the tongue contact each pressure sensor, the detection signal from each pressure sensor is used as a teacher signal,
The contact position and / or contact force with respect to the inner surface of the upper jaw of the tongue is estimated on the basis of the biological signal detected by the biological signal detection element by placing the tongue tip on each pressure sensor, and the contact position and / or Or, based on the teacher signal from the corresponding pressure sensor, until the contact force error is within a predetermined range,
In the above execution estimation process,
9. The neural network according to claim 5, wherein when the tongue tip is brought into contact with the inner surface of the upper jaw, the contact position and / or contact force of the tongue tip is output as an execution estimation state. Method for estimating the stomatognathic movement state.   The neural network performs swallowing movements, opening movements, closing movements, yawning movements, and occlusal movements that occur in addition to the jaw-and-mouth movements to be estimated when the estimation is made based on the contact position and / or contact force of the tongue. The method for estimating a stomatognathic movement state according to claim 9, wherein at least one of the states is estimated simultaneously. In the estimation device of the stomatognathic movement state that estimates the stomatognathic movement state,
Biological signal detection means for detecting a biological signal from a predetermined skin surface of the human body reflecting the movement of muscle related to the jaw and mouth movement, and the state of the jaw and mouth movement based on the biological signal detected by the biological signal detection means A neural network that estimates and outputs
The neural network includes a learning process that learns in advance based on a teacher signal related to the jaw-and-mouth movement until a state error related to the estimation falls within a predetermined range, and a biological body that is newly detected after the learning process ends. An apparatus for estimating a stomatognathic movement state comprising a function of performing an execution estimation process for outputting a state estimated for a signal as an execution estimation state.   The stomatognathic movement includes a motion of a specific motion site, and the state of the stomatognathic motion related to the estimation of the specific motion site is defined by the position and / or force of the specific motion site, The apparatus for estimating the oral cavity movement state according to claim 11.   13. The state of the stomatognathic movement in the specific movement part is a state of contact with another part, and is defined by a contact position and / or a contact force with the other part. For estimating the state of movement of the jaw and mouth   The neural network is configured to estimate all or one of the states of other jaw-and-mouth movements that occur in addition to the jaw-and-mouth movement to be estimated at the time of the estimation based on the position and / or force of the particular movement site. 14. The apparatus for estimating the stomatognathic movement state according to claim 12 or 13, wherein the estimation is also performed for the part.   The biological signal detection means includes a multipoint element group in which a plurality of biological signal detection elements attached to the skin are arranged, and the neural network is configured to calculate a difference between biological signals between the biological signal detection elements of the multipoint element group. 15. The apparatus for estimating a stomatognathic movement state according to any one of claims 11 to 14, wherein processing is performed based on all or part of a combination.   16. The oral cavity according to any one of claims 11 to 15, further comprising movement state identification means for identifying a state of the corresponding stomatognathic movement from the execution estimation state output by the execution estimation process function of the neural network. Motion state estimation device.

Images