JP6994874B2 - Annotation device and noise measurement system - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act General Incorporated Association Acoustical Society of Japan, 2017 Acoustical Society of Japan Spring Research Presentation Proceedings, pp. 795-798, March 1, 2017 2017 Acoustical Society of Japan Spring Research presentation, March 15, 2017

The present invention relates to an annotation device and a noise measurement system.

In noise measurement of environmental sounds that handle a large amount of data, automatic sound source identification is required in order to reduce the labor of selecting the target sound source and other sound sources. A certain sound source type identification device includes a determination means having a neural network, takes data obtained by subbanding the power spectrum distribution of an input sound as an input of the neural network, and generates a sound source type signal as an output of the neural network ( For example, see Patent Document 1).

Japanese Unexamined Patent Publication No. 2001-333304

In order to accurately identify a sound source type with a classifier such as a neural network, a large amount of teacher data (a pair of acoustic data and a sound source type) is required in machine learning. The work of attaching the correct sound source type (label) corresponding to the input data to such input data is called annotation.

In order to prepare the teacher data for the discriminator that identifies the sound source type from the environmental sound, usually, a person plays a long-time recorded data about the environmental sound, listens to the reproduced sound, and a person at each time point. Specify the sound source type of the reproduced sound and label the sound source type. At that time, if it is difficult to understand the situation at the site only by the reproduced sound and the sound source type cannot be specified by one listening, the reproduced sound is repeatedly listened to and the sound source type is specified. Therefore, it takes a huge amount of time and effort to prepare the teacher data for machine learning of the classifier that identifies the sound source type from the environmental sound.

In particular, in the case of reproduced sound in which sounds from a plurality of sound sources overlap, it may not be possible to accurately identify the type of those sound sources only by auditory information. Furthermore, in outdoor propagation, acoustic characteristics may change due to acoustic reflections and diffractions in surrounding buildings, and auditory annotation may not be labeled accurately.

The present invention has been made in view of the above problems, and the time and effort for annotation to generate teacher data for a discriminator that identifies a target sound type such as a sound source type while securing the number of training data. The purpose is to obtain an annotation device and a noise measurement system that reduce the noise.

The annotation device according to the present invention is an annotation device that generates teacher data for machine learning a classifier that outputs output data indicating the classification of the target sound from input data including frequency spectrum data of the acoustic signal of the target sound. It is obtained by a sound collector that collects environmental sounds, an input device that detects real-time user operations that indicate the sound source type by the user when listening to the target environmental sound at the sound collection site, and a sound collector. The sound processing unit that generates the input data of the target sound from the acoustic signal of the environmental sound, and the classification corresponding to the user operation detected by the input device are specified, and the output data indicating the specified classification is given to the input data. Is associated as a label, and a labeling unit is provided in which a pair of input data and output data associated with the input data is used as teacher data. Further, the annotation device according to the present invention has the following configuration (A) or (B). (A) The labeling unit receives the input data generated by the sound processing unit in real time, and associates the specified classification with the input data during the period in which the user operation detected by the input device is detected as a label. (B) A classifier that outputs output data indicating the posterior probabilities of each of a plurality of predetermined classifications from the input data generated by the sound processing unit, and a classifier that is indicated by the output data output by the classifier for each classification. , The display process of displaying the posterior probability as a posterior probability waveform on the display device along the time series and displaying the candidate section on the display device along the section where the posterior probability exceeds a predetermined threshold in the posterior probability waveform. Further including a unit, the input device detects a user operation on the candidate section, and the labeling unit determines one or more candidate sections operated by the user and corresponds to the determined one or more candidate sections. Identify one or more classifications and associate the output data indicating the identified one or more classifications with the input data as labels.

The noise measurement system according to the present invention is provided with the above-mentioned annotating device, machine learning of the classifier is performed by the teacher data generated by the annotating device, and the frequency spectrum of the acoustic signal of the target sound is performed by the machine-learned classifier. From the input data including the data, the output data indicating the sound source type of the target sound is generated.

According to the present invention, since the sound source type of the environmental sound collected at the time of listening at the sound collection site can be recorded, the time for annotation to generate the teacher data for the discriminator that identifies the classification of the target sound such as the sound source type. Annotation devices and noise measurement systems that reduce labor and labor are obtained.

FIG. 1 is a block diagram showing a configuration of an annotation device according to a first embodiment of the present invention. FIG. 2 is a diagram showing an example of a display screen of the first annotation mode. FIG. 3 is a diagram showing an example of a display screen of the second annotation mode. FIG. 4 is an enlarged view of a part of the display screen of FIG. FIG. 5 is a diagram showing an example of a sound source type identification result by a classifier machine-learned with the teacher data generated by the annotation device according to the first embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1.

FIG. 1 is a block diagram showing a configuration of an annotation device according to a first embodiment of the present invention. The annotation device shown in FIG. 1 generates teacher data for machine learning of a classifier that outputs output data indicating the classification of the target sound from the input data including the frequency spectrum data of the acoustic signal of the target sound. In the first embodiment, the classification of the target sound is a sound source type.

The annotation device shown in FIG. 1 includes a sound collecting device 1, a display device 2, an input device 3, a communication device 4, a storage device 5, and an arithmetic processing unit 6.

The sound collecting device 1 is a microphone or the like that collects environmental sounds.

The display device 2 is a liquid crystal display or the like that displays various information to the user.

The input device 3 detects a real-time user operation indicating the sound source type at the time of listening by the user who listened to the above-mentioned environmental sound at the sound collection site. The input device 3 may be a keyboard having hard keys, a keypad, or the like, or may be a touch panel or the like that constitutes soft keys together with a key image displayed on the display device 2. Since the target sound source differs for each sound collection site, the sound source type corresponding to the soft key is set in advance for each sound collection site.

Further, the communication device 4 transmits teacher data and the like to an external device. As the communication device 4, a network interface or a peripheral device interface is used.

The storage device 5 is a non-volatile storage device 5 that stores teacher data and the like. As the storage device 5, a hard disk drive, a flash memory, or the like is used.

The arithmetic processing unit 6 is a computer including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and loads a program stored in the ROM, the storage device 5, and the like into the RAM. However, by executing it on the CPU, it operates as various processing units.

Here, the arithmetic processing unit 6 includes an acoustic processing unit 11, a labeling unit 12, a data output unit 13, a display processing unit 14, a classifier 15, and a learning processing unit 16.

The sound processing unit 11 generates input data among the teacher data from the sound signal of the target sound obtained by the sound collecting device 1. The acoustic processing unit 11 calculates the frequency spectrum data of the acoustic signal and includes it in the input data. For example, as the frequency spectrum data, a short-time Leq (short-time average sound pressure level) for each predetermined bandwidth (for example, 1/3 octave) obtained by a sound level meter having a frequency analysis function is used.

The labeling unit 12 identifies the classification corresponding to the real-time user operation detected by the input device 3, associates the output data indicating the specified classification with the input data as a label, and associates the input data with the labeling unit. The pair with the output data associated with the input data by 12 is set as the teacher data.

The data output unit 13 transmits the teacher data generated by the labeling unit 12 to the outside using the communication device 4 or stores it in the storage device 5.

The display processing unit 14 causes the display device 2 to display various information, key images of soft keys, and the like.

The annotation device according to the first embodiment includes a first annotation mode and a second annotation mode as operation modes, and the user can select either operation mode by using the input device 3.

In the first annotation mode, a plurality of keys for a plurality of predetermined classifications in the input device 3 are used. This key may be a hard key or a soft key. Further, in the first annotation mode, the labeling unit 12 identifies one or a plurality of keys operated by the user, identifies one or a plurality of classifications corresponding to the specified one or a plurality of keys, and identifies one. Alternatively, the output data indicating multiple classifications is associated with the input data as a label. In the first annotation mode, the classification corresponding to the key is continuously specified during the period when the key is pressed.

FIG. 2 is a diagram showing an example of a display screen of the first annotation mode.

As shown in FIG. 2, in the first annotation mode, various time-series data from the current time to a predetermined time (for example, 3 minutes) in the past are displayed on the display device 2 at predetermined time intervals (for example, 1 second). Will be updated. For example, as shown in FIG. 2, the time-series data 51 of the noise level, the time-series data 52 of the frequency spectrum, and the time-series data 53 of the labeling result of a predetermined sound source type are displayed. Further, a soft key layout 54 including soft keys corresponding to a predetermined sound source type is displayed. In the time-series data 53, in the display area corresponding to each sound source type, a specific color is added to the period during which the key corresponding to the sound source type is pressed.

Further, in the first annotation mode, the labeling unit 12 detects the pressing of one or a plurality of soft keys in the soft key layout 54, and the soft keys are pressed with the sound source type corresponding to the detected soft keys as a label. Associate with the input data of the period. Then, the display processing unit 14 repeats at predetermined time intervals to acquire the time-series data 51 of the noise level and the time-series data 52 of the frequency spectrum from the sound processing unit 11, and at the time of the labeling result of the predetermined sound source type. The series data 53 is acquired from the labeling unit 12, and they are displayed on the display device 2 as shown in FIG.

On the other hand, in the second annotation mode, the classifier 15 outputs output data indicating the posterior probabilities (values from 0 to 1) of each of a plurality of predetermined classifications from the input data generated by the sound processing unit 11. , The display processing unit 14 causes the display device 2 to display posterior probabilities as posterior probability waveforms in chronological order for each classification (here, sound source type) indicated by the output data output by the classifier 15. At the same time, the display device 2 displays the candidate section along the section in which the posterior probability exceeds a predetermined threshold value (for example, 0.25) in the posterior probability waveform.

FIG. 3 is a diagram showing an example of a display screen of the second annotation mode. FIG. 4 is an enlarged view of a part of the display screen of FIG.

As shown in FIG. 3, in the second annotation mode, various time-series data from the current time to a predetermined time (for example, 3 minutes) in the past are displayed on the display device 2 at predetermined time intervals (for example, 1 second). Will be updated. For example, as shown in FIG. 3, time-series data 61 of noise level, time-series data 62 of frequency spectrum, and posterior probability waveform 63 of each sound source type are displayed. Further, as shown in FIG. 4, adjacent to the display areas 63a to 63g of the posterior probability waveform 63 of each sound source type, the display areas 64a to 64g of the candidate section corresponding to each sound source type are secured, and the posterior probability waveform 63 is secured. Candidate intervals are displayed corresponding to the intervals in which the posterior probabilities exceed a predetermined threshold in the probability waveform. In FIG. 4, the candidate section 65 is displayed for the sound source type "automobile", the candidate section 66 is displayed for the sound source type "ambulance", and the candidate section 67 is displayed for the sound source type "train". The candidate section 68 is displayed for the type "small bird", and the candidate section 69 is displayed for the sound source type "crow".

Then, in the second annotation mode, the input device 3 detects a user operation for the candidate sections 65, 66, 67, 68, 69 with, for example, a touch panel, and the labeling unit 12 is one or a plurality of candidates operated by the user. Specify the section, specify one or more classifications corresponding to the specified one or more candidate sections, and input the output data indicating the specified one or more classifications (sound source type in FIGS. 3 and 4) as a label. Associate with data.

At this time, the classification (here, sound source type) corresponding to the candidate section is associated as a label with respect to the input data in the time between the start point time and the end point time of the specified candidate section.

The classifier 15 has the same configuration as the classifier in which the teacher data generated by the annotation mode device is used for machine learning (in the case of a deep neural network, the number of hidden layers and the number of nodes in each layer are the same. It has a certain configuration). For example, a deep neural network is used for the classifier 15. For example, the deep neural network has two hidden layers, the input layer is provided with 33 nodes corresponding to frequencies, and the hidden layer of the first stage is provided with 20 nodes. The hidden layer of the next stage is provided with 10 nodes, and the output layer is provided with 55 nodes corresponding to the sound source types.

The learning processing unit 16 performs machine learning of the classifier 15 based on the teacher data generated by the labeling unit 12.

Next, the operation of the annotation device according to the first embodiment will be described.

First, according to the user operation on the input device, each processing unit sets the operation mode to either the first annotation mode or the second annotation mode. The user listens to the target sound at the installation location of the annotation device, and performs an operation on the annotation device according to the classification of the specified target sound.

In the first annotation mode, the labeling unit 12 receives the input data (input data of the teacher data) generated by the sound processing unit 11 in real time, and further, when the key press on the input device 3 is detected. , The classification corresponding to the key (in this case, the sound source type) is specified, the period during which the key press continues (that is, the start time and end time of the key press) is specified, and the specified classification is used for the input data of that period. As a label.

In this way, the pair of input data and output data (ie, the specified classification) is considered as one teacher data set.

Further, as shown in FIG. 2, the display device 2 displays the classification input by the user as time series data 53 in the first annotation mode.

On the other hand, in the second annotation mode, the classifier 15 calculates the posterior probabilities of each classification for the input data in real time, and the display processing unit 14 calculates the posterior probabilities of each classification with respect to the input data, as shown in FIGS. 3 and 4. The probability waveform 63 is displayed on the display device 2, it is determined whether the posterior probability at each time point exceeds a predetermined threshold, and the candidate sections 65 to 69 corresponding to the period in which the posterior probability exceeds the predetermined threshold are set. , Displayed according to the posterior probability waveform 63.

The labeling unit 12 receives the input data (input data of the teacher data) generated by the sound processing unit 11 in real time, and when the input device 3 detects that the candidate section is pressed, the labeling unit 12 receives the input data (input data among the teacher data). The classification corresponding to the candidate section (here, the sound source type) is determined, the period from the start point to the end point of the candidate section is specified, and the specified classification is associated with the input data of that period as a label. As described above, the second annotation mode facilitates the work by the user even for the target sound having a shorter duration than the first annotation mode.

In this way, the pair of input data and output data (ie, the specified classification) is considered as one teacher data set.

As described above, according to the first embodiment, the sound processing unit 11 generates input data (input data among the teacher data) from the sound signal of the target sound obtained by the sound collecting device 1. Then, the labeling unit 12 identifies the classification corresponding to the user operation detected by the input device 3, associates the output data indicating the specified classification with the input data as a label, and associates the input data with the labeling unit. The pair with the output data associated with the input data by 12 is set as the teacher data.

As a result, in order to create teacher data, the user only has to press a key or a candidate section while listening to the target sound while checking the target sound source, and generates teacher data for the discriminator that identifies the sound source type. Saves time and effort for annotation.

FIG. 5 is a diagram showing an example of a sound source type identification result by a classifier machine-learned with the teacher data generated by the annotation device according to the first embodiment. As shown in FIG. 5, the posterior probability of the sound source type is high in the vicinity of each peak of the noise level, and the sound source type is identified.

Embodiment 2.

The noise measurement system according to the second embodiment of the present invention includes the annotation device according to the first embodiment, and machine learning of the classifier is performed by the teacher data generated by the annotation device as described above, and the machine learning is performed. The classifier generates output data indicating the sound source type of the target sound from the input data including the frequency spectrum data of the acoustic signal of the target sound.

This makes it possible to identify a noise source or the like.

It should be noted that various changes and modifications to the above-described embodiments will be apparent to those skilled in the art. Such changes and modifications may be made without departing from the intent and scope of the subject and without diminishing the intended benefits. That is, it is intended that such changes and amendments are included in the claims.

For example, in the first embodiment, the above-mentioned input data may include sound source information such as the direction of the sound source as seen from the sound collecting device 1.

Further, in the same configuration as the noise detection system according to the second embodiment, the above-mentioned target sound is defined as a sound emitted from a specific device, and the above-mentioned target sound is classified as an abnormal noise cause type. The annotation device according to the first embodiment may be applied to an abnormal noise detection system.

Further, in the above-described first and second embodiments, the teacher data is transmitted from the annotation device or the noise measurement system to the server via the network, and the server performs machine learning of the above-mentioned classifier, which is obtained by machine learning. The parameters of the classifier may be transmitted from the server to the annotation device or noise measurement system, and the parameters may be applied to the classifier in the annotation device or noise measurement system.

The present invention is applicable, for example, to the automatic generation of teacher data for a classifier for classifying features of a target sound.

1 Sound collector 2 Display device 3 Input device 11 Sound processing unit 12 Labeling unit 14 Display processing unit 15 Discriminator 16 Learning processing unit

Claims (5)

In an annotation device that generates teacher data for machine learning of a classifier that outputs output data indicating the classification of the target sound from input data including frequency spectrum data of the acoustic signal of the target sound.
A sound collector that collects environmental sounds and
An input device that detects real-time user operations that indicate the sound source type by the user when listening to the environmental sound, and
An acoustic processing unit that generates the input data from the acoustic signal of the target sound obtained by the sound collector, and the acoustic processing unit.
The classification corresponding to the user operation detected by the input device was specified, the output data indicating the specified classification was associated with the input data as a label, and the input data was associated with the input data. A labeling unit whose teacher data is a pair with the output data,
Equipped with
The labeling unit receives the input data generated by the sound processing unit in real time, and the classification specified in the input data during the period in which the user operation detected by the input device is detected is described. Associate as a label,
Annotation device featuring. The input device comprises a plurality of keys for a plurality of predetermined classifications.
The labeling unit identifies one or more keys operated by the user, identifies one or more classifications corresponding to the identified one or more keys, and indicates the identified one or more classifications. Associate the output data as a label with the input data,
1. The annotation device according to claim 1. In an annotation device that generates teacher data for machine learning of a classifier that outputs output data indicating the classification of the target sound from input data including frequency spectrum data of the acoustic signal of the target sound.
A sound collector that collects environmental sounds and
An input device that detects real-time user operations that indicate the sound source type by the user when listening to the environmental sound, and
An acoustic processing unit that generates the input data from the acoustic signal of the target sound obtained by the sound collector, and the acoustic processing unit.
The classification corresponding to the user operation detected by the input device was specified, the output data indicating the specified classification was associated with the input data as a label, and the input data was associated with the input data. A labeling unit whose teacher data is a pair with the output data,
A classifier that outputs output data indicating the posterior probabilities of each of a plurality of predetermined classifications from the input data generated by the sound processing unit.
For each of the classifications indicated by the output data output by the classifier, the posterior probability is displayed on the display device as a posterior probability waveform in chronological order, and the posterior probability waveform is displayed as the posterior probability waveform. A display processing unit for displaying a candidate section on the display device along a section whose probability exceeds a predetermined threshold is provided .
The input device detects a user operation for the candidate section and detects the user operation.
The labeling unit determines one or more candidate sections operated by the user, identifies one or more classifications corresponding to the determined one or more candidate sections, and identifies the one or more classifications. Associating the output data indicating the above with the input data as a label,
Annotation device featuring. The annotation device according to claim 2 or 3, further comprising a learning processing unit that performs machine learning of the classifier based on the teacher data generated by the labeling unit. The annotation device according to claim 1 or 3 is provided.
Machine learning of the classifier is performed using the teacher data generated by the annotation device, and the machine-learned classifier indicates the sound source type of the target sound from the input data including the frequency spectrum data of the acoustic signal of the target sound. Producing output data,
A noise measurement system featuring.

Images