WO2020218129A1

WO2020218129A1 - Determination system, separating device, determination method, and program

Info

Publication number: WO2020218129A1
Application number: PCT/JP2020/016643
Authority: WO
Inventors: 拓司池田; 伸和長嶋; 悠介杉浦; 継河合
Original assignee: トヨタ紡織株式会社
Priority date: 2019-04-22
Filing date: 2020-04-16
Publication date: 2020-10-29
Also published as: JP2020177021A

Abstract

The present invention enables highly accurate sound-based determination in a manufacturing process. A determination system (100) includes: a sound acquisition device (3) that acquires sound including a mating sound and dark noise and that generates sound data; a separating device (4) that separates the component of the mating sound from the sound data by using a learned mating model, thereby generating mating sound data; and a determination device (5) that determines whether or not mating is successful by using the mating sound data.

Description

Judgment system, separator, judgment method, and program

The present invention relates to a determination system or the like that makes a determination based on the sound generated in the manufacturing process of a product.

Conventionally, a technique for detecting an abnormality using sound has been known. For example, Patent Document 1 below discloses a system for determining whether or not there is an abnormality in a structure based on tapping sound data generated when the structure is hit.

Japanese Patent Publication "Japanese Patent Laid-Open No. 2018-13348" (published on January 25, 2018)

In the technique of Patent Document 1, the determination accuracy is lowered when the difference between the normal tapping sound data and the abnormal tapping sound data is small, so that the applicable range of the technique is limited. For example, in the manufacturing process of a certain product, when an operator connects a harness, he / she hears a fitting sound of the harness, and if the sound is quiet, it is determined that the connection is not performed normally. Since the difference between the normal one and the abnormal one is small in such a fitting sound except for the loudness of the sound, whether or not the connection is normally performed even when the technique of Patent Document 1 is used. Is difficult to judge with high accuracy.

Further, in the manufacturing process of a product, when an attempt is made to detect a sound that serves as a judgment criterion for a predetermined judgment item such as the above-mentioned fitting sound, background noise (also referred to as environmental sound) included in the collected sound is generated. It may interfere with the judgment.

One aspect of the present invention has been made in view of the above problems, and an object of the present invention is to provide a determination system or the like capable of making a determination based on sound in a manufacturing process with high accuracy.

In order to solve the above problems, the determination system according to one aspect of the present invention includes a sound acquisition device that acquires a sound including a predetermined sound and background noise generated in a product manufacturing process and generates sound data. , The predetermined sound component is separated from the sound data generated by the sound acquisition device by using a trained model constructed by machine learning based on the teacher data generated from the sound data including the predetermined sound. It includes a separation device that generates sound data, and a determination device that determines predetermined determination items related to the manufacturing process using the predetermined sound data.

In addition, the separation device according to one aspect of the present invention acquires sound data generated from a sound including a predetermined sound and background noise generated in a product manufacturing process in order to solve the above problems. Using the acquisition unit and a trained model constructed by machine learning using the teacher data generated from the sound data including the predetermined sound, the component of the predetermined sound is obtained from the sound data acquired by the sound data acquisition unit. It is provided with a separation unit that separates and generates predetermined sound data.

Further, the determination method according to one aspect of the present invention is a determination method executed by one or a plurality of devices in order to solve the above-mentioned problems, and is a predetermined sound and background noise generated in the manufacturing process of the product. Using the sound data generation step of acquiring the sound including and and generating the sound data, and the trained model constructed by machine learning with the teacher data generated from the sound data including the predetermined sound, the above sound data. A separation step of separating the predetermined sound component from the sound data generated in the generation step to generate the predetermined sound data, and a determination step of determining a predetermined determination item related to the manufacturing process using the predetermined sound data. including.

According to one aspect of the present invention, it is possible to perform a sound-based determination in the manufacturing process with high accuracy.

It is a block diagram which shows the outline of the determination system which concerns on Embodiment 1 of this invention. It is a figure which shows the example of the manufacturing process which is the object of determination of the said determination system. It is a figure which shows an example of the generation method of the fitting sound data. It is a flowchart which shows an example of the determination method executed by the said determination system. It is a block diagram which shows an example of the main part structure of the separation apparatus which concerns on Embodiment 2 of this invention. It is a figure which shows the configuration example of the determination system which concerns on Embodiment 3 of this invention, and the example of the determination method executed by the determination system.

[Embodiment 1]
[Overview of judgment system]
The outline of the determination system 100 according to the present embodiment will be described with reference to FIGS. 1 and 2. The determination system 100 is a system that performs determination based on sound in the manufacturing process of a product. FIG. 1 is a block diagram showing an outline of the determination system 100. FIG. 2 is a diagram showing an example of a manufacturing process to be determined by the determination system 100.

In the manufacturing process shown in FIG. 2, the sheet A1 in which the connector A2 is not connected is sequentially conveyed on the manufacturing line L in the direction of the white arrow in the figure. The seat A1 is, for example, a seat (seat) for a vehicle. The operator manually connects the connector A2 of the sheet A1 transported to the vicinity of the operator. The sheet A1 to which the connector A2 is connected is conveyed to the next process along the production line L.

Connector A2 is an electrical component for connecting electric wires to each other. This electric wire may be a bundle of a plurality of electric wires and cables such as a harness. Since the fitting sound is generated when the connector A2 is correctly connected, the operator can recognize from the fitting sound whether or not the connector A2 is correctly connected. However, when the connector A2 is connected but not completely connected in a semi-fitted state, no fitting sound is generated. However, even when the fitting sound is not generated, the operator mistakenly recognizes that the connector is connected correctly due to the influence of ambient noise or the like, and the connector A2 passes through the sheet A1 in the semi-fitted state. There is also. If the connector A2 is not correctly connected, the sheet A1 may be shipped as a product with a problem of poor energization.

According to the determination system 100, it is possible to determine whether or not the connector A2 is correctly fitted based on the fitting sound of the connector A2 of the sheet A1. Therefore, the determination system 100 can prevent the above-mentioned problem from occurring, or at least reduce the probability that the problem will occur.

As shown in FIG. 1, the determination system 100 includes a system control device 1, a storage device 2, a sound acquisition device 3, a separation device 4, a determination device 5, an output device 6, an image pickup device 7, and a reading device 8. ing.

The system control device 1 is a device that controls the operation of each device included in the determination system 100. Specifically, the system control device 1 stores various data in the storage device 2. Further, the system control device 1 instructs the sound acquisition device 3 to start and end sound acquisition. Further, the system control device 1 instructs the separation device 4 to execute the separation process.

The storage device 2 is a device that stores various information generated by the determination system 100. For example, in the example of FIG. 1, the input sound data, the fitting sound data, and the background noise data are stored in the storage device 2 in association with the product information. These sound data will be described below. The product information is information about the product to be determined, and may include, for example, the identification information of the sheet A1 and the identification information of the production line L of the product.

The sound acquisition device 3 is a device that acquires the sound generated in the manufacturing process and generates input sound data to be input to the separation device 4. The sound acquisition device 3 is typically a microphone. The sound acquisition device 3 transmits the generated input sound data to the separation device 4. The input sound data may be transmitted by connecting a transmission device (not shown) to the sound acquisition device 3 by wire or wirelessly and via the transmission device, or by providing the sound acquisition device 3 with a communication unit and connecting the communication unit. You may go through. The sound acquired by the sound acquisition device 3 includes not only fitting sound but also background noise.

Here, when making a judgment using a sound such as a fitting sound generated by the work of the operator, the distance or position between the position where the sound is generated (the position where the operator is working) and the sound acquisition device If the relationship fluctuates, it becomes difficult to obtain a sound with stable volume and sound quality. In addition, the determination accuracy of the determination system 100 may decrease accordingly.

Therefore, when making a determination using the fitting sound, it is preferable that the sound acquisition device 3 is a device worn by the operator during work, as shown in the example of FIG. As a result, it is possible to prevent the distance and positional relationship between the position where the sound is generated and the sound acquisition device from fluctuating significantly, so that sound with stable volume and sound quality can be acquired, and the determination accuracy is maintained. can do. In the example of FIG. 2, the sound acquisition device 3 is attached to the cuffs of the operator, which is the optimum attachment position for acquiring the fitting sound generated at the operator's hand. Of course, the mounting position of the sound acquisition device 3 may be any position as long as it can acquire the fitting sound, and is not limited to the cuffs.

The separation device 4 is a device that separates the fitting sound component from the input sound data generated by the sound acquisition device 3 and generates the fitting sound data. Although the details will be described later, a trained model constructed by machine learning using teacher data generated from sound data including the fitting sound is used to generate the fitting sound data.

The determination device 5 is a device that determines whether or not the connector has been fitted correctly by using the fitting sound data generated by the separation device 4. For example, when the loudness of the generated sound is different between when the connector A2 is completely fitted and when the connector A2 is half-fitted, the determination device 5 determines the fitting sound data generated by the separation device 4. It may be determined whether or not the fitting is performed correctly based on the loudness of the fitting sound specified from. For example, the determination device 5 may determine that the fitting is performed correctly when the loudness of the fitting sound is equal to or more than the threshold value, and may determine that the fitting is not performed correctly when the loudness is less than the threshold value. The determination result by the determination device 5 is output by the output device 6.

The output device 6 may be any device as long as it can output the determination result, and may be, for example, a display device that displays an image showing the determination result. FIG. 2 shows an example in which the output device 6 is a display device integrally configured with the determination device 5. In this case, when it is determined that the fitting is performed correctly, the output device 6 displays a character or symbol (for example, "OK") indicating that, and it is determined that the fitting is not performed correctly. In that case, characters or symbols (for example, "NG") indicating that may be displayed. Further, the output device 6 may be an audio output device that outputs a voice indicating a determination result, or a light emitting device or the like that indicates the determination result in a lit state.

The image pickup device 7 is a device for capturing an image. As shown in FIG. 2, the image pickup apparatus 7 is installed toward an operator who performs a connector fitting operation. As a result, the imaging device 7 can capture a moving image of the operator performing the fitting operation. Although the details will be described later, this moving image is input to the system control device 1, and the system control device 1 determines the timing at which the sound acquisition device 3 starts acquiring sound based on the moving image.

The reading device 8 is a device that acquires product information of the product to be determined. For example, when the product information is recorded as a two-dimensional code or a three-dimensional code attached to the sheet A1, the reading device 8 may be a reading device for the code. The reading device 8 transmits the acquired product information to the system control device 1.

As described above, the determination system 100 is based on the sound acquisition device 3 that acquires the sound including the fitting sound and the background noise and generates the input sound data, and the teacher data generated from the sound data including the fitting sound. Using the learned model constructed by machine learning, the separation device 4 that separates the components of the fitting sound from the input sound data generated by the sound acquisition device 3 to generate predetermined sound data, and the fitting sound data are used. A determination device 5 for determining whether or not the fitting of the connector A2 is correctly performed is included. According to this configuration, since the judgment is performed using the fitting sound data generated by separating the fitting sound component from the input sound data, it is possible to reduce the influence of background noise and perform a highly accurate judgment. become.

[Method of generating mating sound data]
A method of generating fitting sound data by the separation device 4 will be described with reference to FIG. FIG. 3 is a diagram showing an example of a method of generating fitting sound data. As described above, in the manufacturing process of the sheet A1, a fitting sound is generated when the operator connects the connector A2, and the sound including the fitting sound and the background noise is acquired by the sound acquisition device 3. .. Then, the sound acquisition device 3 converts the acquired sound to generate input sound data, and transmits the input sound data to the separation device 4.

The separator 4 that has received the input sound data first converts the received input sound data into frequency data to generate a spectrogram. A Fourier transform may be used for the conversion to frequency data.

Next, the separation device 4 inputs the generated spectrogram into the trained model and outputs the spectrogram of the mating sound. This trained model is constructed by machine learning so that the fitting sound data can be separated from the sound data including the fitting sound.

The above trained model is preferably constructed by supervised learning. In this case, the spectrogram generated from the sound data including the fitting sound may be used as the teacher data. Then, the components of the fitting sound included in this spectrogram are labeled. By performing machine learning using a large number of such teacher data, it is possible to construct a trained model that separates the mating sound spectrogram from the spectrogram containing the mating sound spectrogram and the background noise spectrogram. ..

Note that the spectrogram used as the teacher data may include a background noise component in addition to the fitting sound component. Such a spectrogram can be generated, for example, from sounds collected in an actual manufacturing process.

Finally, the separation device 4 converts the spectrogram of the fitting sound output from the trained model into the fitting sound data which is the sound data instead of the frequency data, and thus the generation of the fitting sound data is completed. The inverse Fourier transform may be used for the conversion to the fitting sound data. Further, the separation device 4 may generate a background noise spectrogram by removing the components of the mating sound spectrogram from the spectrogram generated from the input sound data, and may also generate background noise data from the background noise spectrogram. .. When a determination device for determining the correctness of fitting is used based on the spectrogram, the separation device 4 may output the spectrogram of the fitting sound output from the learned model as the fitting sound data.

When a sound is reproduced from the fitting sound data generated as described above, the sound becomes a sound having a loudness proportional to the loudness of the fitting sound included in the input sound data. Therefore, the determination device 5 can determine whether the fitting is correct or semi-fitting by using the fitting sound data generated by the separation device 4.

As described above, the separation device 4 generates a spectrogram which is frequency data from the input sound data generated by the sound acquisition device 3, inputs the spectrogram into the trained model, and outputs a spectrogram of the fitting sound. Then, the separation device 4 restores the spectrogram of the fitting sound into the sound data and generates the fitting sound data.

According to the above configuration, the fitting sound can be separated based on the feature points of the frequency data. Further, according to the above configuration, since the frequency data is restored to the sound data to generate the fitting sound data, the determination device 5 can perform the determination using the sound data (data that is not the frequency data). This configuration is suitable for a determination system that makes a determination using an apparatus such as the determination device 5 that makes a determination based on the loudness of the sound.

[Processing flow]
The process executed in the determination system 100, that is, the determination method will be described with reference to FIG. FIG. 4 is a flowchart showing an example of a determination method executed by the determination system 100. Note that in FIG. 4, the processing of the storage device 2, the output device 6, the image pickup device 7, and the reading device 8 is omitted.

<< Processing executed by the system control unit >>
In S1, the system control device 1 determines whether or not the sound acquisition device 3 starts sound collection. For example, the system control device 1 analyzes the moving image captured by the imaging device 7 to determine the direction of the operator's line of sight reflected in the moving image, and the direction is set to a predetermined direction (for example, the direction of the connector A2). When facing, it may be determined that the sound acquisition device 3 starts collecting sound. Further, for example, when the system control device 1 analyzes the moving image, detects the worker's hand reflected in the moving image, and the worker's hand makes a predetermined movement (for example, a movement of grasping the connector A2). In addition, it may be determined that the sound acquisition device 3 starts collecting sound. In addition to this, for example, the timing at which the sound acquisition device 3 starts collecting sound is determined from the posture of the worker and the orientation of the body (facing the seat A1, tilting the body forward toward the connector A2, etc.). May be good. The processing of S1 is continuously performed until it is determined to start sound collection (YES in S1), and when it is determined to be YES in S1, the processing proceeds to S2.

In S2, the system control device 1 instructs the sound acquisition device 3 to start sound collection. As described above, it is preferable that the system control device 1 causes the sound acquisition device 3 to start acquiring the sound when the work of generating the fitting sound is started. This is because the sound acquisition is started when the work of generating the fitting sound is started, so that the useless sound (the sound that cannot include the fitting sound) occupying the sound acquired by the sound acquisition device 3 is started. This is because the fitting sound can be efficiently separated by keeping the ratio low.

Then, in S3, the system control device 1 instructs the sound acquisition device 3 to end the sound collection. The process of S3 may be executed, for example, when a predetermined time elapses after the sound collection is started in S2. The predetermined time is combined with the time of the sound data that is the source of the teacher data used for constructing the trained model used by the separation device 4. For example, when a trained model constructed by using a spectrogram generated from sound data for 3 seconds as teacher data is used, the predetermined time is also set to 3 seconds.

From the viewpoint of outputting the determination result at an early stage, the above-mentioned predetermined time satisfies the condition that the sound acquisition device 3 continues to collect sound when the connector A2 is fitted by the operator. , Preferably as short a time as possible. For example, if the time from the operator looking at the connector A2 to completing the connection of the connector A2 is about several seconds, the predetermined time may be set to about several seconds to about ten and several seconds. In this case, a trained model corresponding to this predetermined time may be used.

In S4, the system control device 1 instructs the separation device 4 to separate the fitting sound data. As a result, the processing of the system control device 1 is completed. Although not shown, the system control device 1 may acquire product information from the reading device 8. Then, after the separation of the fitting sound data is completed, the system control device 1 acquires the fitting sound data, the background noise data, and the input sound data from the separating device 4, and corresponds the sound data with the product information. It may be attached and stored in the storage device 2.

<< Processing executed by the sound acquisition device >>
In S11, the sound acquisition device 3 starts acquiring the sound around the sound acquisition device 3 and generating the input sound data according to the instruction from the system control device 1 in S2. In the following S12, the sound acquisition device 3 ends the acquisition of the surrounding sound of the sound acquisition device 3 and the generation of the input sound data according to the instruction from the system control device 1 of the S3. Then, the sound acquisition device 3 transmits the input sound data generated in the processes of S11 to S12 to the separation device 4.

<< Processing executed by the separator >>
In S21, the separation device 4 receives the input sound data transmitted by the sound acquisition device 3 in S13. In the following S22, the separation device 4 separates the fitting sound data from the input sound data received in S21 according to the instruction from the system control device 1 transmitted in S4. The method of separating the fitting sound data is as described with reference to FIG. Then, in S23, the separation device 4 transmits the fitting sound data separated in S22 to the determination device 5. Although not shown, the separation device 4 may transmit the fitting sound data to the system control device 1 and store it in the storage device 2. The same applies to the input sound data and the background noise data.

<< Processing executed by the judgment device >>
In S31, the determination device 5 receives the fitting sound data transmitted by the separation device 4 in S23. In the following S32, the determination device 5 determines whether or not the fitting is performed correctly using the fitting sound data received in S31. Then, in S33, the determination device 5 causes the output device 6 to output the determination result of S32.

It is preferable that the determination device 5 immediately outputs the determination result in order to prevent the product that has not been fitted correctly from being conveyed to the next process. The determination device 5 may output the determination result only when it is determined that the fitting is not performed correctly. In this case, the output device 6 may be a buzzer that emits a warning sound, a warning light, or the like.

Further, when the determination device 5 determines that the fitting is not performed correctly, the determination device 5 controls the product transfer device to stop the transfer so that the product that is not fitted correctly is not transferred to the next process. It may be. The position on the transport line L at which the product is stopped is arbitrary, but it may be stopped at a predetermined position in consideration of workability. Further, in this case, the determination system 100 may also determine whether or not the fitting has been performed correctly in the refitting work by the operator. Then, the determination device 5 may restart the transportation of the product when it is determined that the fitting has been performed correctly.

≪Summary≫
As described above, the determination method of FIG. 4 includes a sound data generation step (S11 to S12) for acquiring a sound including a fitting sound and a background noise and generating sound data, and a sound generated in the sound data generation step. A separation step (S22) for separating the components of the fitting sound from the data to generate the fitting sound data, and a determination step (S32) for determining the correctness of the fitting using the fitting sound data are included.

Therefore, the judgment based on the fitting sound can be performed with high accuracy. In the above example, the execution subject of each step is a different device, but one device may execute two or more steps of each of the above steps. For example, when the separation device 4A of the second embodiment described later is used, the sound data generation step is executed by the sound acquisition device 3 as in the present embodiment, but the separation step and the determination step are executed by the separation device 4A. It will be.

[Embodiment 2]
Other embodiments of the present invention will be described below. For convenience of explanation, the same reference numerals will be added to the members having the same functions as the members described in the above embodiment, and the description will not be repeated.

FIG. 5 is a block diagram showing an example of the main configuration of the separation device 4A according to the present embodiment. The separation device 4A is a device having the functions of the system control device 1, the storage device 2, the separation device 4, the determination device 5, and the output device 6 in the determination system 100 of the first embodiment. Therefore, when the separation device 4A is used, for example, the determination system can be constructed by four devices, the separation device 4A, the sound acquisition device 3, the image pickup device 7, and the reading device 8. As described above, the function of the determination system 100 can be realized by various device configurations.

As shown in FIG. 5, the separation device 4A includes a control unit 41 that controls and controls each part of the separation device 4A, a storage unit 42 that stores various data used by the separation device 4A, and the separation device 4A with other devices. It includes a communication unit 43 for communication and a display unit 44 for displaying an image. Further, the control unit 41 includes a start determination unit 411, a sound data acquisition unit 412, a separation unit 413, a fitting correctness determination unit 414, and a determination result notification control unit 415. Further, the separation unit 413 includes a spectrogram generation unit 413A, a spectrum separation unit 413B, and a sound data conversion unit 413C. Then, the input sound data 421, the background noise data 422, and the fitting sound data 423 are stored in the storage unit 42.

The start determination unit 411 determines whether or not the sound acquisition device 3 starts acquiring the sound. For example, the start determination unit 411 analyzes the moving image captured by the imaging device 7 as in the system control device 1 of the first embodiment, determines the direction of the operator's line of sight in the moving image, and determines the direction of the operator's line of sight. May determine that the sound acquisition device 3 starts collecting sound when is oriented in a predetermined direction.

The sound data acquisition unit 412 acquires input sound data generated from a sound including a fitting sound and background noise. Specifically, when the start determination unit 411 determines that the sound acquisition is to be started, the sound data acquisition unit 412 instructs the sound acquisition device 3 to start the sound acquisition. Further, the sound data acquisition unit 412 instructs the sound acquisition device 3 to end the sound acquisition when a predetermined time elapses after starting the sound acquisition. Then, the sound data acquisition unit 412 acquires the sound data acquired in response to the above instruction from the sound acquisition device 3 and stores it in the storage unit 42 as the input sound data 421.

Each function of the start determination unit 411 and the sound data acquisition unit 412 is the same as the function of the system control device 1 of the first embodiment. Therefore, the system control device 1 of the first embodiment can also be realized by providing the system control device 1 with a control unit (for example, a processor) including a start determination unit 411 and a sound data acquisition unit 412.

The separation unit 413 separates the fitting sound component from the input sound data and generates the fitting sound data. As will be described below, this function is realized by the spectrogram generation unit 413A, the spectrum separation unit 413B, and the sound data conversion unit 413C.

The spectrogram generation unit 413A converts the input sound data acquired by the sound data acquisition unit 412 into frequency data to generate a spectrogram. Further, the spectrum separation unit 413B inputs the spectrogram generated by the spectrogram generation unit 413A into the trained model, and outputs a spectrogram of the fitting sound. The trained model is the same as that described in the first embodiment. Then, the sound data conversion unit 413C converts the spectrogram of the fitting sound output from the learned model into the fitting sound data, and stores it in the storage unit 42 as the fitting sound data 423.

Note that the spectrum separation unit 413B may generate a background noise spectrogram from the spectrogram generated by the spectrogram generation unit 413A by removing the spectrogram component of the fitting sound. In this case, the sound data conversion unit 413C may generate background noise data from the background noise spectrogram and store it in the storage unit 42 as background noise data 422.

The functions of each part included in the separation part 413 are the same as the functions of the separation device 4 of the first embodiment. Therefore, the separation device 4 of the first embodiment can also be realized by providing the separation device 4 with a control unit (for example, a processor) including a spectrogram generation unit 413A, a spectrum separation unit 413B, and a sound data conversion unit 413C.

The fitting correctness determination unit 414 determines whether or not the fitting has been performed correctly using the fitting sound data generated by the sound data conversion unit 413C. Then, the determination result notification control unit 415 causes the display unit 44 to display the determination result of the fitting correctness determination unit 414. Each of these functions is the same as the function of the determination device 5 of the first embodiment. Therefore, the determination device 5 of the first embodiment can also be realized by providing the determination device 5 with a control unit (for example, a processor) including a fitting correctness determination unit 414 and a determination result notification control unit 415.

As described above, the separation device 4A is fitted from the input sound data by using the sound data acquisition unit 412 that acquires the input sound data generated from the sound including the fitting sound and the background noise and the trained model. It includes a separation unit 413 that separates sound components and generates fitting sound data.

According to the separation device 4A, the fitting correctness determination unit 414 can make a determination based on the fitting sound with high accuracy. As in the first embodiment, the determination device 5 may be used for determination. In this case, the separation device 4A transmits the fitting sound data generated by the sound data conversion unit 413C to the determination device 5. As a result, the separation device 4A can make the determination device 5 perform the determination based on the fitting sound with high accuracy, as in the separation device of the first embodiment.

Further, the separation device 4 of the first embodiment also acquires the input sound data generated from the sound including the fitting sound and the background noise, and separates the fitting sound component from the input sound data by using the trained model. To generate fitting sound data. Therefore, it can be said that the separation device 4 of the first embodiment also includes the sound data acquisition unit 412 and the separation unit 413. Therefore, the same effect as that of the separation device 4A can be obtained by the separation device 4.

[Embodiment 3]
Other embodiments of the present invention will be described below with reference to FIG. For convenience of explanation, the same reference numerals will be added to the members having the same functions as the members described in the above embodiment, and the description will not be repeated.

FIG. 6 is a diagram showing a configuration example of the determination system 100 according to the present embodiment and an example of a determination method executed by the determination system 100. As shown in the figure, the determination system 100 of the present embodiment does not include the determination device 5 and the output device 6 as compared with the determination system 100 of the first embodiment (see FIG. 1), and the separation device 4 is the separation device 4B. It differs in that it has changed to.

The separation device 4B is a device having the functions of the determination device 5 and the output device 6 in addition to the functions of the separation device 4 of the first embodiment. Although not shown, the separation device 4B includes a control unit similar to the separation device 4A, and the control unit includes a sound data acquisition unit 412, a separation unit 413, a fitting correctness determination unit 414, and a determination unit shown in FIG. The result notification control unit 415 is included. Although not shown, the separation device 4B includes a storage unit 42, a communication unit 43, and a display unit 44, similarly to the separation device 4A of FIG.

[Processing flow]
The process executed in the determination system 100 of the third embodiment will be described. Since the processes executed by the system control device 1 and the sound acquisition device 3 and the processes of S21 to S22 are the same as the processes described with reference to FIG. 4, the description will not be repeated here.

In S23'(determination step), the fitting correctness determination unit 414 of the separation device 4B uses the fitting sound data separated from the input sound data by the separation unit 413 in S22 to determine whether or not the fitting is performed correctly. judge. Then, in S24', the determination result notification control unit 415 outputs the determination result of S23'to the display unit 44. The determination result notification control unit 415 transmits the input sound data acquired by the sound data acquisition unit 412 in S21 and the fitting sound data separated in S22 to the system control device 1 together with the determination result of S23'. It may be stored in the storage device 2.

[Modification example]
The determination system 100 may perform determination based on the sound generated in the manufacturing process of the product, and the sound serving as the determination standard is not limited to the fitting sound. Further, the determination items of the determination system 100 are also arbitrary. For example, when the seat A1 is an electric power seat, the determination system 100 may determine whether or not there is an abnormality in the seat A1 based on the operating sound. Further, in this case, the determination system 100 may determine the presence or absence of an abnormality in the motor based on the operating noise of the motor for driving the seat frame.

Of course, the target product is not limited to vehicle seats and their parts, but may be a product that has technical significance in making a judgment based on the sound generated in the manufacturing process. For example, the determination system 100 may determine the presence or absence of an abnormality in the manufacturing equipment based on the sound generated from the manufacturing equipment of the product. In this way, the determination system 100 can also be used for maintenance of manufacturing equipment. The sound generated from the manufacturing equipment of the product may be generated at the time of manufacturing, or may be generated (or artificially generated) at the time of non-manufacturing such as maintenance of the manufacturing equipment. Good.

When changing the sound that serves as the judgment standard, the trained model used by the

separation devices

4, 4A, or 4B may be set according to the sound that serves as the judgment standard. As a result, the sound component that serves as the judgment criterion can be separated as sound data from the input sound data that includes the sound that serves as the judgment criterion and the background noise, so that the influence of background noise can be suppressed and a highly accurate judgment can be made. be able to.

Further, in the above embodiment, an example of determining the timing for starting the sound acquisition by the sound acquisition device 3 based on the image of the operator captured by the image pickup device 7 has been described, but the timing for starting the sound acquisition has been determined. The method is not limited to this example. For example, when a specific sound (for example, a contact sound or a joining sound of the connector A2) is generated immediately before fitting the connector A2, the generation of input sound data including the fitting sound is started with the detection of the sound as a trigger. You may. Further, for example, if the worker is asked to utter at the start of the fitting operation, the generation of the input sound data including the fitting sound can be started with the detection of the utterance as a trigger. In addition to this, for example, it is detected that the sheet A1 to be the target of the work of connecting the connector A2 next reaches a predetermined position on the production line L, and the detection triggers an input sound including a fitting sound. You can also start generating data.

Further, the

separators

4, 4A, and 4B generate a frequency mask that allows the frequency component of the fitting sound of the connector to pass through and does not allow the frequency component of the background noise to pass through, and uses this frequency mask to fit from the input sound data. Fitting sound data may be generated by separating the sound components. In this case, the

separators

4, 4A, and 4B input a spectrogram including connector fitting noise and background noise into a trained model pre-built by machine learning. As a result, the trained model is made to output a frequency mask that allows the frequency component of the fitting sound of the connector to pass through and does not pass the frequency component of the background noise.

Then, the

separators

4, 4A, and 4B pass the spectrogram through the frequency mask to obtain a spectrogram containing the frequency component of the fitting sound of the connector and removing the frequency component of the background noise. After that, the spectrogram may be converted into the fitting sound data in the same manner as in the above-described embodiment.

The above trained model may be constructed by supervised learning using teacher data. Specifically, the above teacher data uses a spectrogram including the fitting sound of the connector as input data, and outputs a frequency mask that passes the frequency component of the fitting sound of the connector and does not pass the frequency component of background noise (correct answer). It may be the teacher data as (data).

[Example of realization by software]
The functions of each device (particularly the system control device 1, the separation device 4, and the judgment device 5) included in the determination system 100, and the control blocks (particularly each unit included in the control unit 41) of the

separation devices

4A and 4B are integrated. It may be realized by a logic circuit (hardware) formed in a circuit (IC chip) or the like, or may be realized by software.

In the latter case, the system control device 1, the separation device 4, the

separation devices

4A, 4B, and the determination device 5 include a computer that executes a program instruction that is software that realizes each function. The computer includes, for example, one or more processors and a computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, in addition to a “non-temporary tangible medium” such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (RandomAccessMemory) for expanding the above program may be further provided. Further, the program may be supplied to the computer via an arbitrary transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. It should be noted that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present invention.

1 System control device 3

Sound acquisition device

4, 4A, 4B Separator 5 Judgment device 100 Judgment system 412 Sound data acquisition unit 413 Separation unit

Claims

A sound acquisition device that acquires sound including a predetermined sound and background noise generated in the manufacturing process of a product and generates sound data,
Using a trained model constructed by machine learning based on teacher data generated from sound data including the predetermined sound, the predetermined sound component is separated from the sound data generated by the sound acquisition device to obtain the predetermined sound. A separator that generates data and
A determination system including a determination device for determining a predetermined determination item related to the manufacturing process using the predetermined sound data.
The sound acquisition device is a device worn by a worker in the manufacturing process during work.
The determination system according to claim 1, wherein the predetermined sound is a sound generated by the work of the worker.
The first or second claim, wherein the sound acquisition device further includes a system control device that starts the sound acquisition when the work of generating the predetermined sound is started in the manufacturing process. Judgment system.
The separation device generates frequency data from the sound data generated by the sound acquisition device, inputs the frequency data to the trained model to output the frequency data of the predetermined sound, and outputs the frequency data to the sound data. The determination system according to any one of claims 1 to 3, wherein the predetermined sound data is generated by restoring the data.
A sound data acquisition unit that acquires sound data generated from sounds that include a predetermined sound and background noise generated in the manufacturing process of a product, and a sound data acquisition unit.
Using a trained model constructed by machine learning based on teacher data generated from sound data including the predetermined sound, the predetermined sound component is separated from the sound data acquired by the sound data acquisition unit. A separation device including a separation unit that generates sound data.
A program for operating a computer as the separation device according to claim 5, wherein the computer functions as the sound data acquisition unit and the separation unit.
A determination method performed by one or more devices.
A sound data generation step that acquires sound including a predetermined sound and background noise generated in the manufacturing process of a product and generates sound data,
Using a trained model constructed by machine learning with teacher data generated from sound data including the predetermined sound, the predetermined sound component is separated from the sound data generated in the sound data generation step. Separation steps to generate sound data and
A determination method including a determination step of determining a predetermined determination item related to the manufacturing process using the predetermined sound data.