KR100762183B1 - Tactile stimulator apparatus and method for deaf-mute - Google Patents

Abstract

The present invention relates to an intelligent tactile stimulation device and method for the hearing impaired, which detects environmental sounds generated in the surroundings and converts the information into tactile senses, which are different senses, and compares them with previously stored information. Similar information is recognized and the recognized information is transmitted to the wearer in the form of a tactile stimulus using a stimulation device.

Deafness, tactile stimulation, environmental sound, stimulation pattern

Description

Intelligent tactile stimulation device and method for the deaf {TACTILE STIMULATOR APPARATUS AND METHOD FOR DEAF-MUTE}

1 is a block diagram of an intelligent tactile stimulation device for the hearing impaired according to the present invention.

FIG. 2A is a diagram showing a square wave stimulation pattern, and FIG. 2B is a diagram showing an analog wave stimulation pattern.

3 is a flowchart illustrating an intelligent tactile stimulation method for the disabled according to the present invention.

4 to 7 is a view showing the appearance of the tactile stimulation device according to an embodiment of the present invention.

The present invention relates to a hearing aid device for the hearing impaired, and more particularly, to the hearing impaired intelligent tactile stimulation device for the hearing impaired to detect and transmit the environmental sound generated in the surroundings and converts the information into a different sense of tactile sense And to a method.

Although it is impossible to recognize environmental sound or convey information through voice to the hearing impaired person, it is possible to grasp the current situation to some extent by grasping the surrounding situation according to the ability of the person even without the sound within the field of vision. However, the sound of a horn, a fire alarm, or the like, or an environmental sound such as a ringtone, which requires an urgent situation occurring outside the field of view cannot be recognized at all.

An overview of the existing hearing aids for the hearing impaired introduced at home and abroad are as follows.

The wireless handheld stimulation device developed by sonic Alert, Inc. is a system that receives a specific sound generated from a specific device such as a telephone or a fire alarm in a house in a wireless form and notifies the wearer that the sound is generated in the form of vibration. You can cope with some of the sounds in your home.

Dow E & C (equipment specialized facilities for the disabled) releases equipment that notifies the hearing impaired visually by connecting the fire alarm device and lighting device inside the building so that the lighting device emits strong light in case of fire. Doing.

In addition, a device that divides general sound into various sound ranges and stimulates the body using a two-channel or seven-channel vibrotactile device is known in the US Audiological Eng. It is released by Corp.

In addition, a device for storing a vocabulary of a word in advance and receiving a voice word occurring in a nearby place, recognizing it through a comparison, and converting it into a tactile vibration form is already developed (Domestic Patent No. 2002-0321645). Reference).

However, because these technologies place spatial constraints on the conveying of specific meanings, they have spatial limitations in the role of the actual wearer's hearing aid, and in terms of conveying information about the sound in other ways, The focus is on simply conveying the 'characteristics of the sound itself'. Therefore, in order to recognize the sound change characteristics in the form of different tactile senses and determine the original meaning, a long time comparison learning process is required.

To secure this point, a tactile stimulator using speech recognition has been developed recently to assist the hearing-impaired person in conversation or life.However, real voice alone does not provide an understanding of what is happening outside the field of view. Therefore, the wearer's mouth recognition is more effective than the wearer's recognition of the result of speech recognition through the sense of touch.

As described above, the conventional assisted devices for the hearing impaired are not limited to a specific place or can not determine the meaning of the environmental sound, and even using voice recognition can greatly help real life. There is no.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object of the present invention is to provide an intelligent tactile stimulation device and method for the hearing impaired that can recognize environmental sound information occurring not only in the field of view but also outside the field of view.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In describing the operating principles of the preferred embodiments of the present invention in detail with reference to the accompanying drawings, when it is determined that the detailed description of the related well-known functions or configurations may unnecessarily obscure the gist of the present invention, the detailed description will be given. Will be omitted. Among the terms used in the present invention, 'environmental sound' means a sound of a vehicle horn, a fire alarm, or an ambient sound, such as a ringing sound, not a human voice and a human voice. It can be configured to recognize the environmental sound information for a specific specified sound.

1 is a block diagram of an intelligent tactile stimulation device for the hearing impaired according to the present invention.

Referring to Figure 1, the intelligent tactile stimulation device of the present invention is largely environmental sound input unit 10, environmental sound recognition tactile converter 20 and. And a tactile stimulus 30 and an external computer system 40.

The environmental sound input unit 10 receives environmental sounds from the outside.

The environmental sound recognition tactile change unit 20 converts the sound recognition and the tactile conversion of the environmental sound input from the environmental sound input unit 10 by using the sound feature point information and the tactile stimulation pattern information corresponding to the sound feature point information. And outputs a predetermined tactile stimulation pattern.

The tactile stimulation unit 30 transmits the tactile stimulation pattern output from the environmental sound recognition tactile conversion unit 20 to the wearer as an actual stimulus, and a vibrating stimulator and an electrical stimulation stimulator are used.

The external computer system 40 may generate a tactile stimulation pattern corresponding to a predetermined environmental sound by embedding a predetermined tactile stimulation pattern generation program, and the intelligent tactile stimulation device of the present invention may download the generated tactile stimulation pattern. It provides a tactile stimulation pattern for various environmental sounds.

Specifically, the environmental sound input unit 10 includes a microphone 12 for receiving only a sound generated in the entire area or a sound in a specific direction, and a signal band of the environmental sound input from the microphone 12 as an audible frequency band. A band pass filter 14 for limiting and removing direct current components, and an amplifier 16 for amplifying a signal output from the band pass filter 14.

The microphone 12 uses an omnidirectional microphone for receiving sound generated over an entire area and a directional microphone for receiving only sound in a specific direction.

The amplifying unit 16 may be an operational amplifier. When amplifying using a single operational amplifier, the amplification degree increases, so that the bandwidth is narrowed. Preferably, the amplification rate is lowered to widen the bandwidth, and then amplifies again. Amplify the sound signal using several operational amplifiers to have a wide bandwidth.

The environmental sound recognition tactile converter 20 includes an analog / digital converter 21, a feature point extractor 23, an environmental sound recognizer 25, a tactile stimulus pattern extractor 27, and a first memory. A unit 24, a second memory unit 26, a digital / analog converter 29, and a control unit 22 for controlling them are configured. Here, the first memory unit 24 stores sound feature point information, and the second memory unit 26 stores tactile stimulation pattern information.

The analog / digital converter 21 converts the signal output from the amplifier 16 into a digital signal. For example, a 12-bit analog to digital converter can be used.

The feature point extractor 23 extracts a sound feature point of a digitally converted signal through the analog / digital converter 21, and always receives a digital signal from an environment sound input unit, thereby comparing an internal comparator (not shown). When the environmental sound signal is input over a certain size (eg 50dB), the feature points for it are extracted.

Preferably, the feature point extraction is performed using a non-stationary continuous wavelet transform capable of extracting the feature point in consideration of both the loudness and the frequency characteristic over time.

The environmental sound recognizer 25 recognizes the environmental sound corresponding to the extracted sound feature point by comparing the feature point of the signal extracted from the feature point extractor 23 with the sound feature point information stored in the first memory unit 24. . That is, a specific environmental sound is recognized by comparing the feature points of the environmental sound obtained by the feature point extractor 23 with the feature points of the environmental sound stored in the first memory in advance, for example, by using a neural network. Build a recognition network.

The tactile stimulus pattern extractor 27 extracts a tactile stimulus pattern corresponding to the environmental sound recognized by the environmental sound recognizer 25 from the tactile stimulus pattern information stored in the second memory unit 26.

The digital / analog converter 29 converts the tactile stimulus pattern output from the tactile stimulus pattern extractor 27 into an analog signal and transmits the analogue signal to the tactile stimulus 30.

The controller 22 controls the overall functionality of the device and mediates communication with an external computer system.

The tactile stimulation pattern in the present invention may be a square wave stimulation pattern that considers the transmission and stop of the stimulus, the duration of the stimulus, or an analog wave stimulation pattern that considers the transmission and stop of the stimulus, the intensity of the stimulus, and the duration of the stimulus. have.

FIG. 2A is a diagram showing a square wave stimulation pattern, in which a stimulus of 0.1 seconds, 0.2 seconds, and 0.3 seconds is transmitted with a 0.1 second stop time.

FIG. 2B is a diagram showing an analog wave type stimulation pattern, wherein unit time 0.1 seconds, stimulus intensity step 5, stimulation sequence 0, 2, 5, 4, 1, 0, 0, 2, 2, 2, 0, and It can be seen that the stimulus propagation time is 1.2 seconds.

As shown in Figure 2a, the tactile stimulation pattern using a square wave generates a signal in consideration of three of the tactile stimulation frequency, the interval between the stimulus signal, and the transmission output of the stimulus signal for the optimal tactile transmission. The stimulus signal is composed of a tactile stimulus time, a tactile stimulus waiting time, and a stimulus intensity, and is controlled using an adjusting device so that the user can adjust the optimal stimulus intensity.

As shown in FIG. 2B, the tactile stimulation pattern using the analog wave form generates a signal in consideration of three types of tactile stimulation frequencies, intervals between stimulus signals, and grades of stimulus intensities. It is composed of the change of the stimulus intensity of the tactile signal and the propagation time of the stimulus, and the degree of change of the stimulus intensity is controlled by the control of the controller, and the difference of each stimulus intensity can be distinguished through the stimulus adaptive learning training of the wearer. The stimulus intensity can be divided into several stages, and can be implemented in various stages according to the wearer's learning ability.

3 is a flowchart illustrating an intelligent tactile stimulation method for the disabled according to the present invention.

First, when the environmental sound is input from the outside (S10), the preprocessing process for the input environmental sound is performed (S12). The omnidirectional microphone is used to detect the signal outside the field of view to maximize the area of the environmental sound input, and the preprocessing process is performed to remove the noise and the band that is not necessary for the recognition of the electric environmental sound signal received from the microphone. do.

The pretreatment process consists of a filtering process and an amplification process.

In the filtering process, a bandpass filter is used to limit the frequency range of the signal and remove noise. The environmental sound is characterized by having a direct current component when input through a microphone. In addition, since the audible frequency band that humans can perceive by hearing is 20 to 20 kHz, a signal other than the audible frequency band that is not necessary for sound recognition is not necessary. Therefore, the band pass filter is used to limit the signal band of the environmental sound to the audible frequency band and to remove the direct current component.

In the amplification process, for example, an operational amplifier is used to amplify an acoustic signal.

Next, a sound recognition tactile conversion process is performed, which includes the feature point extraction (S14), the environmental sound recognition (S16), and the stimulus pattern extraction (S18).

The amplified environmental sound signal is converted into a digital signal, and the feature point of the digitally converted environmental sound signal is extracted (S14). At this time, if the environmental sound signal of a predetermined size (for example, 50dB) is inputted to extract the feature point for it. Preferably, the feature point extraction is performed using an abnormal continuous wavelet transform capable of extracting the feature point in consideration of both the loudness and the frequency characteristic over time.

When the extraction of the feature points for the environmental sounds is completed, environmental sound recognition is performed (S16). The environmental sound recognition is performed by comparing the feature points of the extracted environmental sounds with the feature points of the environmental sounds in the pre-stored environmental sound feature point information. Do this.

When the environmental sound recognition is completed, the tactile stimulation pattern corresponding to the recognized environmental sound is extracted from the tactile stimulation pattern information stored in advance (S18).

Finally, the tactile stimulation pattern extracted in step S18 is delivered to the vibration type stimulator or the electrical stimulation type stimulator and transmitted to the wearer as the actual stimulus (S20).

4 to 7 show the appearance of the tactile stimulation device according to an embodiment of the present invention, and is divided into a necklace type, an arm band type (or wrist type), a waist type, and a headset type according to the attachment type.

Referring to Figure 4, the necklace-type tactile stimulation device 100 is configured to be worn in the form of a necklace, the case 101, the connection terminal 102 for the connection with the external computer and the size of the stimulation for the magnetic pole size 103 is provided, the necklace 104 is provided with a tactile stimulator 106 in the part in contact with the wearer's neck and a microphone 105 for receiving the environmental sound from the outside when worn in the center.

Referring to FIG. 5, the arm band type (or wrist type) tactile stimulation device 200 is attached to an arm or a wrist by using an arm band or a wrist band 206, and a stimulus size on the front of the case 201. Stimulus size control unit 203 and the microphone 204 for receiving the environmental sound from the outside is provided, the side of the case 201 is provided with a connection terminal 202 for connecting to an external computer, the case The back of the 201 is provided with a tactile stimulator 205 so that the tactile stimulus can be transmitted to the arm or wrist when worn.

Referring to FIG. 6, the waist type tactile stimulation device 300 is attached to the waist by a fixing holder (not shown) or the like, and the case 301 has a connection terminal 302 for connection with an external computer. And a stimulus size control unit 303 for stimulus size, a tactile stimulator 304 for transmitting a tactile stimulus to the waist when worn, and a microphone 305 for receiving environmental sounds from the outside. .

Referring to FIG. 7, the headset-type tactile stimulation device 400 is worn in the form of a general headphone. The tactile stimulator 404 is connected to the headband 405 and the hairband 405 to transmit a stimulus to an ear part. Consists of a tactile stimulator housing 401, the tactile stimulator housing 401b of one of the tactile stimulator housing 401, the connection terminal 402 for connecting to an external computer, and the size of the stimulus for the stimulus size adjustment The unit 403, and a microphone 406 for receiving the environmental sound from the outside is provided.

According to the present invention, the wearer with a hearing impairment selects a meaningful environmental sound capable of recognizing the surroundings, stores feature points for the sound information in advance, and recognizes the sound information when the corresponding sound information is recognized. By transmitting to the tactile stimulus, which is a third sense, it provides an effect of recognizing environmental sound information among environmental sounds generated not only in the field of view but also outside the field of view.

Claims (9)

In the intelligent tactile stimulation device for the disabled to allow the wearer with hearing impairment to recognize the surrounding situation, Environmental sound input unit for receiving the environmental sound from the outside; Environment sound for extracting sound feature point information by using an abnormal continuous wavelet transform considering the magnitude and frequency characteristics of the environment sound to recognize the environment sound and outputting a tactile stimulus pattern corresponding to the sound feature point information A recognition tactile converter; A tactile stimulator for delivering a tactile stimulus pattern output from the environmental sound recognition tactile converter to a wearer as an actual stimulus; And an controller for controlling the operation of the environment sound input unit, the environment sound recognition tactile conversion unit, and the tactile stimulation unit. The method according to claim 1, A tactile stimulation pattern generation program may be built in to generate tactile stimulation patterns corresponding to environmental sounds, and the intelligent tactile stimulation device may provide tactile stimulation patterns for various environmental sounds by downloading the generated tactile stimulation patterns. Intelligent tactile stimulation device for the disabled, characterized in that it further comprises an external computer system capable of. The method according to claim 1, The environmental sound input unit includes a microphone for receiving only a sound generated in an entire area or a sound in a specific direction, a band pass filter for limiting a signal band of the environmental sound input from the microphone to an audible frequency band and removing a direct current component; An amplifier for amplifying the signal output from the bandpass filter; The environmental sound recognition tactile converter includes a first memory unit for storing the sound feature point information, a second memory unit for storing the tactile stimulus pattern information, and an analog for converting a signal output from the amplifier into a digital signal The digital conversion unit, a feature point extracting unit for extracting sound feature points of the digitally converted signal, and the feature point of the signal extracted from the feature point extracting unit by comparing the sound feature point information stored in the first memory unit with the extraction; Extracting the tactile stimulation pattern corresponding to the environmental sound recognized by the environmental sound recognition unit from the tactile stimulation pattern information stored in the second memory unit; Analyzes the tactile stimulation pattern extraction unit and the tactile stimulation pattern output from the tactile stimulation pattern extraction unit. Intelligent tactile stimulation device for the disabled, characterized in that consisting of a digital / analog conversion unit for converting into a log signal and then transmitted to the tactile stimulation unit. The method according to claim 1, The tactile stimulation pattern is a disabled person, characterized in that the square wave stimulation pattern in consideration of the transmission and stop of the stimulus, the duration of the stimulus or analog wave type stimulation pattern in consideration of the transmission and stop of the stimulus, the intensity of the stimulus, the duration of the stimulus Intelligent tactile stimulation device. delete delete delete In the intelligent tactile stimulation method for the disabled to make the wearer with hearing impairment aware of the surrounding situation, A first step of performing a preprocessing process of limiting the signal band of the environmental sound to an audible frequency band, filtering and amplifying the filtered signal by removing a DC component when the environmental sound is input through the environmental sound input unit; The environmental sound recognition tactile converter grasps the sound feature point information with respect to the environment sound, and compares the sound feature point information with the sound feature point information stored in the first memory unit to recognize the environment sound, and responds to the sound feature point information. Outputting stored tactile stimulation pattern information; And the third step of transmitting the tactile stimulation pattern as a real stimulus to the wearer through the tactile stimulation unit. The method according to claim 8, The second step, Converting the amplified signal into a digital signal; Extracting sound feature points of the digitally converted signal; Recognizing environmental sounds corresponding to the extracted sound feature points by comparing the feature points of the signal extracted from the feature point extractor with previously stored sound feature point information; Extracting tactile stimulation patterns corresponding to the recognized environmental sounds from tactile stimulation pattern information stored in advance; Intelligent tactile stimulation method for the disabled, further comprising the step of converting the extracted tactile stimulation pattern into an analog signal.

Images