JP7246337B2 - Computer system and work estimation method - Google Patents

Description

TECHNICAL FIELD The present invention relates to a work estimation method based on a worker's speech.

A system is known in which an algorithm (model) for understanding a user's utterance intention is implemented using speech data obtained from the user (see, for example, Patent Document 1 and Non-Patent Document 1). In this specification, the system is described as an utterance intention estimation system.

Patent Literature 1 discloses that a speech intention estimation device includes speech acquisition means for acquiring speech data of a user's utterance, feature acquisition means for acquiring an acoustic feature that is an acoustic feature of the utterance, and the acoustic feature an intention estimating means for estimating the intention of the user's utterance from the quantity, wherein the intention estimating means may be configured to be capable of estimating the intention of the user's utterance even by using text of the utterance; When the text of the user's utterance can be extracted, the intention is estimated using the text of the utterance, and when the text of the user's utterance cannot be extracted from the voice data or the utterance intention cannot be estimated from the text. It is also preferable to estimate intention using acoustic features."

Non-Patent Document 1 describes correcting the current intention understanding result in consideration of past utterance contents.

JP 2018-169494 A

Koichiro Yoshino, "Understanding User's Intention from Spoken Utterance Sequence", Journal of Institute of Electronics, Information and Communication Engineers, Vol.101 No.9 pp.896-901, September 2018

It is assumed that an utterance intention estimation system will be used to estimate the content and results of work performed by workers in factories or sites where work such as product manufacturing or equipment inspection is performed.

When estimating the content and result of work performed by a worker, it is difficult to estimate the content and result of work from the utterance intention of each utterance. Therefore, as described in Non-Patent Document 1, it is necessary to estimate the utterance intention using the utterance sequence.

The combination of utterances uttered by workers during work (combination of utterance content and utterance order) may differ even for the same work. Since it is generally difficult to generate models corresponding to combinations of various utterances, conventional models are fixed to specific combinations of utterances.

The present invention realizes an utterance intention estimation system capable of estimating with high accuracy the content and result of work performed by a worker for input of various combinations of utterances.

A representative example of the invention disclosed in the present application is as follows. That is, a computer system for estimating work performed by a worker based on utterances, comprising at least one computer having a processor and a memory connected to the processor, wherein the processor receives the converting speech data into text, storing the text in the memory, calculating an utterance interval between the utterances from a plurality of the time-series continuous voice data, storing the utterance interval in the memory; generating time-series data of utterances in which the plurality of texts are arranged in chronological order and time-series data of utterance intervals in which the utterance intervals are arranged in chronological order; is stored in the memory, and based on the time-series data of the speech and the time-series data of the speech interval, the content and result of the work performed by the worker are estimated, and the result of the estimation is output.

According to the present invention, a computer system (speech intention estimation system) can highly accurately estimate the details and results of work performed by a worker in response to input of various combinations of speech. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

1 is a diagram showing an example of the configuration of an utterance intention estimation system of Example 1; FIG. 4 is a diagram showing the flow of intention understanding model generation processing executed by the computer of Example 1. FIG. 4 is a flowchart showing an example of intention understanding model generation processing executed by the computer of the first embodiment; 4 is a diagram showing an example of time-series data input to the computer of Example 1. FIG. 4 is a diagram showing an example of time-series data input to the computer of Example 1. FIG. 4 is a diagram showing an example of time-series data input to the computer of Example 1. FIG. 4 is a diagram showing an example of the data structure of learning data generated by the computer of Example 1. FIG. 4 is a diagram showing the flow of work estimation processing executed by the computer of Example 1. FIG. 5 is a flow chart showing an example of work estimation processing executed by the computer of the first embodiment; 4 is a diagram showing an example of the data structure of intermediate output information generated by the computer of Example 1; FIG. 13 is a flow chart showing an example of intention understanding model generation processing executed by the computer of Example 2. FIG. FIG. 10 is a diagram showing an example of time-series data of attribute labels input to the computer of Example 2; FIG. 10 is a diagram showing an example of the data structure of utterance pattern information generated by the computer of Example 2; FIG. 10 is a diagram showing an example of a method of calculating an utterance interval of an utterance pattern in Example 2;

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention should not be construed as being limited to the contents of the examples described below. Those skilled in the art will easily understand that the specific configuration can be changed without departing from the idea or gist of the present invention. In the configurations of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and overlapping descriptions are omitted. The notations such as “first”, “second”, “third”, etc. in this specification and the like are attached to identify the constituent elements, and do not necessarily limit the number or order. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not limited to the positions, sizes, shapes, ranges, etc. disclosed in the drawings and the like.

FIG. 1 is a diagram showing an example of the configuration of the utterance intention estimation system according to the first embodiment.

The utterance intention estimation system is composed of a computer 100 and a microphone 101 . Note that the number of computers 100 and microphones 101 may be two or more.

A microphone 101 is a device for collecting voices uttered by workers. The microphone 101 generates audio data from collected audio and transmits the audio data to the computer 100 . The microphone 101 may be fixed in the space where the worker is working, or may be carried by the worker. The voice data includes a time stamp indicating at least one of the speech start time and the speech end time.

The computer 100 estimates the content and result of the work performed by the worker using the voice data. The computer 100 comprises a processor 110 , a main storage device 111 , a secondary storage device 112 and a connection interface 113 . Each piece of hardware is connected to each other via an internal bus.

The connection interface 113 is an interface for connecting with an external device. The connection interface 113 is, for example, a network interface and an I/O interface.

The processor 110 executes programs stored in the main memory device 111 . The processor 110 operates as a functional unit (module) that implements a specific function by executing processing according to a program. In the following description, when processing is described with a functional unit as the subject, it means that the processor 110 is executing a program that implements the functional unit.

The main storage device 111 stores programs executed by the processor 110 and data used by the programs. The main storage device 111 also includes a work area that is temporarily used by the program. The main storage device 111 is, for example, a DRAM (Dynamic Random Access Memory) or the like. Programs stored in the main storage device 111 will be described later.

Note that the programs and data stored in the main storage device 111 may be stored in the secondary storage device 112 . In this case, processor 110 reads programs and data from secondary storage device 112 and stores them in main storage device 111 .

The secondary storage device 112 permanently stores data. The secondary storage devices 112 are, for example, HDDs (Hard Disk Drives) and SSDs (Solid State Drives). The data stored in secondary storage device 112 will be described later.

Here, programs and data held by the computer 100 will be described.

The secondary storage device 112 stores speech recognition model information 130 and intent understanding model information 131 .

The speech recognition model information 130 is definition information of a speech recognition model for estimating the specific content of an utterance from speech data. The speech recognition model of Example 1 is, for example, NN (Neural Network). Features of speech data are input to the nodes that make up the network. By using the speech recognition model, text indicating the utterance content is output.

The intent understanding model information 131 is definition information of an intent understanding model for estimating the details and results of the work performed by the worker using time-series data of utterances. The intent understanding model of Example 1 is, for example, RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), CNN (Convolution Neural Network), and the like. Features calculated from unit utterances and utterance intervals are input to the nodes that make up the network. Unit utterances are, for example, sentences and words. In Example 1, the unit utterance is a sentence. The utterance interval is the time interval from when one utterance is made until when the next utterance is made. By using the intent understanding model, a combination of work content and results is output.

The main memory device 111 stores programs that implement the speech recognition unit 120 , the work estimation unit 121 , the learning data generation unit 122 , and the learning unit 123 .

The speech recognition unit 120 uses a speech recognition model to generate text representing the specific content of the utterance from the speech data.

The task estimation unit 121 estimates the content and result of the task using the intention understanding model and the time-series data of the utterance (text).

The learning data generation unit 122 generates learning data used by the learning unit 123 . In Example 1, the learning data generation unit 122 generates learning data for generating an intent understanding model. Note that the learning data generation unit 122 may generate learning data for generating a speech recognition model.

The learning unit 123 executes learning processing for generating a model using learning data. In Example 1, the learning unit 123 executes learning processing for generating an intent understanding model. Note that the learning unit 123 may execute learning processing for generating a speech recognition model.

As for each functional unit of the computer 100, a plurality of functional units may be combined into one functional unit, or one functional unit may be divided into a plurality of functional units for each function.

FIG. 2 is a diagram showing the flow of intention understanding model generation processing executed by the computer 100 of the first embodiment. FIG. 3 is a flowchart showing an example of intent understanding model generation processing executed by the computer 100 of the first embodiment. 4A, 4B, and 4C are diagrams showing examples of time-series data input to the computer 100 of the first embodiment. FIG. 5 is a diagram showing an example data structure of learning data 200 generated by the computer 100 of the first embodiment.

Calculator 100 receives input of time-series data of utterances (text) for learning, time-series data of utterance intervals for learning, and time-series data of correct labels (step S101). Calculator 100 outputs the received data to learning data generator 122 .

The time-series data of utterances for learning is a data group of utterances organized on the basis of time, work type, worker, location, and the like. Time-series data of utterances for learning is, for example, time-series data 400 as shown in FIG. 4A. The time-series data 400 stores records composed of a sequence 401 and utterance content 402 . There is one record for one utterance. The order 401 is a field that stores the speaking order. The utterance content 402 is a field that stores text.

The time-series data of utterance intervals for learning is a data group of time intervals between utterances included in the time-series data of utterances for learning. Time-series data of utterance intervals for learning is, for example, time-series data 410 as shown in FIG. 4B. The time-series data 400 stores records composed of a sequence 411 and an utterance interval 412 . Order 411 is the same field as order 401 . The utterance interval 412 is a field that stores the time interval between the utterance corresponding to the record and the utterance one chronologically before the utterance. Since there is no utterance before the utterance corresponding to the record whose order 411 is "1", the utterance interval 412 is blank.

The time-series data of correct labels is a data group of correct labels indicating correct outputs (classes) for utterances included in the time-series data of utterances for learning. Time-series data of the correct label is, for example, time-series data 420 as shown in FIG. 4C. The time-series data 420 stores records composed of a sequence 421 and a correct label 422 . Order 421 is the same field as order 401 . Correct label 422 is a field that stores the correct output of the intent understanding model. A value is set in the correct label 422 for each class (combination of work content and result). As shown in FIG. 4C, the intent understanding model outputs one of five classes: paper jam OK, paper jam NG, paper remaining OK, paper remaining NG, and others. Either "0" or "1" is set for each class. A class set to "1" is a correct class.

Calculator 100 is input with a plurality of data sets in which time-series data of utterances for learning, time-series data of utterance intervals for learning, and time-series data of correct labels are grouped together. A plurality of data sets having the same reference may be input.

Note that when time-series data of utterances for learning includes time information, the time-series data of utterance intervals for learning need not be given as an input. In this case, it is possible to generate time-series data of utterance intervals for learning from time-series data of utterances for learning.

Calculator 100 generates learning data 200 using the input time-series data (step S102).

Specifically, the learning data generation unit 122 generates learning data 200 including a record that associates an utterance with an utterance interval and a correct label corresponding to the utterance. Note that the learning data generation unit 122 generates one learning data 200 for one data set.

Here, the learning data 200 will be explained. As shown in FIG. 5 , the learning data 200 includes records composed of a sequence 501 , an utterance interval 502 , utterance content 503 , and a correct label 504 . One record corresponds to one utterance. The order 501 , the speech interval 502 , the speech content 503 , and the correct label 504 are the same fields as the order 401 , the speech interval 412 , the speech content 402 , and the correct label 422 .

The utterance intention estimation system of Example 1 is characterized in that the utterance interval is treated as an input for the intention understanding model. As a result, even if the utterance group includes utterances unrelated to the work, it is possible to accurately estimate the content and result of the work.

Returning to the description of the intent understanding model generation process.

Next, computer 100 executes learning processing using learning data 200 (step S103).

Specifically, the learning unit 123 treats the feature amount calculated from the utterance interval 502 and the utterance content 503 included in the learning data 200 as an input to the node, and intends to obtain an output corresponding to the correct label 504. Generate an understanding model. Since a known method may be used for the learning method, detailed description is omitted.

For example, a vector calculated from an utterance and an utterance interval are input as feature amounts to the nodes that constitute the network corresponding to the intent understanding model. For example, a vector can be calculated from an utterance using Word2vec or the like.

Next, computer 100 saves the learning result (intention understanding model) in intention understanding model information 131 (step S104), and terminates the intention understanding model generation process.

FIG. 6 is a diagram showing the flow of work estimation processing executed by the computer 100 of the first embodiment. FIG. 7 is a flowchart illustrating an example of work estimation processing executed by the computer 100 of the first embodiment. FIG. 8 is a diagram showing an example of the data structure of intermediate output information 800 generated by the computer 100 of the first embodiment.

The computer 100 acquires voice data from the microphone 101 (step S201). It is assumed that computer 100 accumulates voice data for a certain period of time.

Calculator 100 generates a text indicating the specific content of the utterance from the voice data, and calculates the utterance interval (step S202). Specifically, the following processing is executed.

(S202-1) The speech recognition unit 120 selects target speech data from the accumulated speech data.

(S202-2) The speech recognition unit 120 uses the speech recognition model stored in the speech recognition model information 130 to generate text from the target speech data. In addition, the speech recognition unit 120 calculates an utterance interval between the target speech data and the speech data (comparative speech data) preceding the target speech data in the time series.

For example, the speech recognition unit 120 calculates the time from the end time of the speech corresponding to the comparison speech data to the start time of the speech corresponding to the target speech data as the speech interval. Further, the speech recognition unit 120 may calculate the time from the start time of the speech corresponding to the comparative speech data to the start time of the speech corresponding to the target speech data as the speech interval. Note that when there is no comparison voice data, the voice recognition unit 120 calculates the utterance interval as NULL.

(S202-3) The speech recognition unit 120 determines whether or not all the accumulated speech data have been processed. If the processing of all the accumulated voice data has not been completed, the voice recognition unit 120 returns to S202-1 and performs similar processing. When the processing of all the accumulated voice data is completed, the voice recognition unit 120 ends the processing of step S202.

The above is the description of the processing in step S202.

Next, computer 100 generates time-series data of utterances for input (step S203).

Specifically, the speech recognition unit 120 generates input data composed of text and speech intervals. The speech recognition unit 120 generates an input data group arranged in order of utterance as time-series data of utterance for input.

Next, computer 100 executes an estimation process using the intent understanding model and the input time-series data of utterances (step S204). Specifically, the following processing is executed.

(S204-1) The work estimation unit 121 generates the intermediate output information 800. FIG. The intermediate output information 800 includes records composed of a sequence 801 , utterance content 802 , and an output class 803 . One record corresponds to one input data.

The order 801 is a field that stores the order of utterances in the input time-series data of utterances. The utterance content 802 is a field that stores the text included in the input data. The output class 803 is a field for storing the output result of the intention estimation model, and a value is set for each class. In FIG. 8, the output class 803 includes five classes: paper jam OK, paper jam NG, paper remaining OK, paper remaining NG, and others. At this point, the value of each class is set to "0".

(S204-2) The work estimating unit 121 inputs the input data included in the time-series data of utterances for input into the intention understanding model based on the order of utterances. The work estimation unit 121 acquires output results for each input data from the intention understanding model. At this time, the work estimation unit 121 refers to the output class 803 of the record corresponding to the input data, and sets "1" to the class corresponding to the output result.

In Example 1, an utterance interval is input together with the text indicating the utterance. Even if an utterance unrelated to work is included between the utterance for identifying the work type and the utterance for identifying the work result, the intent understanding model considers the utterance interval and can be estimated, the content and results of the work can be estimated with high accuracy. In the example shown in FIG. 8, an utterance not related to "paper jam inspection work" is included between the record whose order 801 is "4" and the record whose order 801 is "8". By using the intention understanding model, it is possible to output correct work contents and results for the record "8".

(S204-3) When the output results of all the input data are obtained, the work estimation unit 121 ends the estimation process.

The above is the description of the processing in step S204.

Next, computer 100 outputs an estimation result based on intermediate output information 800 (step S205), and terminates the work estimation process.

For example, the task estimating unit 121 generates an estimation result in which the output results for which the output class 803 is other than "other" is listed, and presents it to the user.

According to the first embodiment, the utterance intention estimation system uses an intention estimation model whose inputs are utterances and utterance intervals, so that between an utterance for identifying a work type and an utterance for identifying a work result, , even if there are utterances unrelated to the work, the content and result of the work can be estimated with high accuracy. That is, it is possible to realize an utterance intention estimation system that supports various combinations of utterances.

Note that when the unit utterance is a word, the utterance interval is the time interval between words.

Note that a feature amount calculated from sound pressure may be input to the node of the intent understanding model. For example, when the sound pressure of utterances during work is low and the sound pressure of utterances other than work is high, it can be used as a useful feature amount. In the intention understanding model generation process, learning data 200 is generated from time-series data of utterances for learning, time-series data of utterance intervals for learning, time-series data of correct labels, and time-series data of sound pressure for learning. be. In the task estimation process, the speech recognition unit 120 calculates sound pressure from speech data, and generates an input data group composed of text, speech intervals, and sound pressure.

The second embodiment differs in the method of generating learning data. The second embodiment will be described below, focusing on the differences from the first embodiment.

The configuration of the utterance intention estimation system of the second embodiment is the same as that of the first embodiment.

FIG. 9 is a flowchart showing an example of intent understanding model generation processing executed by the computer 100 of the second embodiment. FIG. 10 is a diagram showing an example of time-series data of attribute labels input to the calculator 100 of the second embodiment. FIG. 11 is a diagram showing an example of the data structure of utterance pattern information 1100 generated by the computer 100 of the second embodiment. FIG. 12 is a diagram illustrating an example of a method of calculating an utterance interval of an utterance pattern according to the second embodiment.

Calculator 100 receives inputs of time-series data of utterances for analysis, time-series data of utterance intervals for analysis, and time-series data of attribute labels (step S151). Calculator 100 outputs the received data to learning data generator 122 .

Time-series data of utterances for analysis has the same data structure as time-series data of utterances for learning. Time-series data of utterance intervals for analysis has the same data structure as time-series data of utterance intervals for learning.

An attribute label is information indicating an abstraction intent. The abstract intent is an abstraction of the utterance intent. In the second embodiment, the abstract intention of the utterance intention "confirm paper jam" and the utterance intention "confirm remaining amount of paper" is set to "confirm work type". The abstract intention of the utterance intention "work result OK" is set to "work result OK", and the abstract intention of the utterance intention "work result NG" is set to "work result NG". The abstract intention of the utterance intention that does not correspond to any of the above is set as "other".

Time-series data of attribute labels is, for example, time-series data 1000 as shown in FIG. Time-series data 1000 stores records composed of a sequence 1001 and an attribute label 1002 . Order 1001 is the same field as order 401 .

The attribute label 1002 is a field that stores a value indicating an abstraction intention for each work type. I1 represents "work type confirmation", I2 represents "work result OK", I3 represents "work result NG", and I4 represents "others". Either "0" or "1" is set for each abstract intent. An abstraction intent set to "1" indicates that it is a work type abstraction intent. If the abstract intent of all work types is "other", the abstract intent of the utterance is "other."

In the second embodiment, it is assumed that the number of data sets (time-series data of utterances) is small. In this case, the number of generated learning data 200 is small. In order to generate a highly accurate intent understanding model, it is necessary to prepare a large amount of high-quality training data.

Next, computer 100 performs statistical analysis using time-series data of utterances for analysis, and generates utterance pattern information 1100 based on the analysis results (step S151). Specifically, the following processing is executed.

(S152-1) The learning data generation unit 122 generates an utterance group from the start to the end of work for each work type based on the time-series data of utterances and the time-series data of attribute labels included in each data set. do. An utterance group for each work type is generated for one data set. If the time-series data of utterances included in one data set includes utterances of the same task that have been executed multiple times, multiple utterance groups with the same task type can be found in one data set of utterances for analysis. generated.

For example, in the case of time-series data of utterances for analysis of the data structure shown in FIG. 4A, a group of utterances of records with the order 401 of "2" and "3" is generated for "checking paper jam". In addition, regarding the “work to check the remaining amount of paper”, an utterance group of records whose order 401 is “4” and “5” is generated.

(S152-2) The learning data generator 122 selects a target work type.

(S152-3) The learning data generation unit 122 acquires an utterance group of the target work type from a plurality of data sets, and generates an utterance pattern based on the attribute labels of the utterances included in the utterance group. The utterance pattern is information indicating the transition of abstract intention. For example, a transition from 'work type confirmation' to 'work result OK' is generated as an utterance pattern from records whose order 401 is '2' and '3'.

(S152-4) The learning data generation unit 122 generates utterances corresponding to the abstract intent of the utterance pattern based on the time-series data of the utterance intervals for analysis corresponding to the utterance group of the target work type in the plurality of data sets. Calculate the speech interval. The learning data generation unit 122 generates time-series data of the utterance intervals of the utterance pattern by arranging the utterance intervals in chronological order.

If the number of dialogue groups of the utterance pattern of the target work type is small, the calculation result of another work type having a large number of dialogue groups of the utterance pattern may be used. For example, as shown in FIG. 12, the learning data generation unit 122 calculates the variance of the utterance intervals for the utterance pattern with the other work type, and calculates the utterance interval of the utterance pattern with the target work type based on the calculation result. do. In addition, the above processing is applied only to specific utterance patterns such as an utterance pattern consisting only of "work type confirmation" and "work result OK" or an utterance pattern consisting only of "work type confirmation" and "work result NG". may

(S152-5) The learning data generation unit 122 determines whether or not processing has been completed for all work types. If the processing has not been completed for all work types, the learning data generation unit 122 returns to S152-2 and performs similar processing.

(S152-6) When the processing has been completed for all work types, the learning data generator 122 initializes the utterance pattern information 1100. FIG.

Specifically, the learning data generation unit 122 adds to the utterance pattern information 1100 the same number of records as the generated utterance pattern types. The learning data generation unit 122 sets identification information to the ID 1101 of each record of the utterance pattern information 1100, and sets the same number of columns as work types to each of the probability 1103 and the utterance interval 1104. FIG. Learning data generator 122 sets an utterance pattern to utterance pattern 1102 of each record of utterance pattern information 1100 . “Others (2)” included in the utterance pattern shown in FIG. 11 indicates that two utterances with the abstraction intention “others” are included.

(S152-7) The learning data generator 122 selects a target work type.

(S152-8) The learning data generator 122 calculates the appearance probability of each utterance pattern based on the dialogue group of the target work type. The learning data generation unit 122 sets the appearance probability in the target work type column of the probability 1103 of each record of the utterance pattern information 1100 .

(S152-9) The learning data generator 122 sets the time-series data of the utterance intervals of the utterance pattern of the target work type in the target work type column of the utterance intervals 1104 of each record of the utterance pattern information 1100. FIG. Speech intervals are separated by semicolons.

(S152-10) The learning data generator 122 determines whether or not the processing has been completed for all work types. If the processing has not been completed for all work types, the learning data generation unit 122 returns to S152-7 and performs similar processing. When the processing has been completed for all work types, the learning data generation unit 122 ends the processing of step S152.

The above is the description of the processing in step S152.

Next, computer 100 receives input of assumed utterance data for each work type (step S153). Note that the input of assumed utterance data may be performed in step S101.

The assumed utterance data includes a plurality of records composed of utterances and utterance intentions. Note that the utterance of the utterance intention "other" may be common to each work type.

Next, computer 100 executes learning data generation processing (step S154). Specifically, the following processing is executed.

(S154-1) The learning data generator 122 selects a target work type.

(S154-2) Based on the utterance pattern of the target work type, the utterance interval time-series data of the utterance pattern, and the assumed utterance data, the learning data generation unit 122 generates utterance time-series data, utterance interval time-series data, And a plurality of data sets are generated in which the time-series data of the correct labels are grouped together. The number of data sets generated from each utterance pattern is controlled so as to match the appearance probability ratio.

Time-series data of utterances in one data set can be generated by selecting utterances from assumed utterance data according to utterance patterns. In addition, the time-series data of the utterance intervals of one data set may be generated using the utterance intervals of the utterance pattern as they are, or may be generated using a value obtained by adding perturbation to the utterance intervals of the utterance pattern. . The width of the perturbation can be set based on the probability of speech intervals. The time-series data of the correct labels in one data set can be generated by converting the abstract intentions of the utterance pattern into utterance intentions corresponding to the target work type, and arranging the utterance intentions in chronological order.

(S154-3) The learning data generating unit 122 generates learning data including a plurality of records composed of utterances, correct labels, and utterance intervals using one data set. The processing is the same as the processing in step S102.

(S154-4) The learning data generation unit 122 determines whether or not processing has been completed for all work types. If the processing has not been completed for all work types, the learning data generation unit 122 returns to S154-1 and performs similar processing. When the processing has been completed for all work types, the learning data generation unit 122 ends the processing of step S154.

The above is the description of the processing in step S154.

The processing of steps S103 and S104 is the same as the processing described in the first embodiment.

Since the work estimation process of the second embodiment is the same as the work estimation process of the first embodiment, the description thereof is omitted.

According to the second embodiment, the computer 100 can generate new learning data from a small number of learning data. By increasing the number of learning data, the prediction accuracy of the intent understanding model generated by the learning process can be improved.

In addition, the present invention is not limited to the above-described embodiments, and includes various modifications. Further, for example, the above-described embodiments are detailed descriptions of the configurations for easy understanding of the present invention, and are not necessarily limited to those having all the described configurations. Moreover, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

Further, each of the above configurations, functions, processing units, processing means, and the like may be realized by hardware, for example, by designing a part or all of them using an integrated circuit. The present invention can also be implemented by software program code that implements the functions of the embodiments. In this case, a computer is provided with a storage medium recording the program code, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium storing it constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, A nonvolatile memory card, ROM, or the like is used.

Also, the program code that implements the functions described in this embodiment can be implemented in a wide range of programs or scripting languages such as assembler, C/C++, perl, Shell, PHP, Python, and Java (registered trademark).

Furthermore, by distributing the program code of the software that implements the functions of the embodiment via a network, it can be stored in storage means such as a hard disk or memory of a computer, or in a storage medium such as a CD-RW or CD-R. Alternatively, a processor provided in the computer may read and execute the program code stored in the storage means or the storage medium.

In the above-described embodiments, the control lines and information lines indicate those considered necessary for explanation, and not all control lines and information lines are necessarily indicated on the product. All configurations may be interconnected.

100 computer 101 microphone 110 processor 111 main storage device 112 secondary storage device 113 connection interface 120 speech recognition unit 121 work estimation unit 122 learning data generation unit 123 learning unit 130 speech recognition model information 131 intention understanding model information 200 learning data 400, 410, 420, 1000 time-series data 800 intermediate output information 1100 utterance pattern information

Claims (8)

A computer system for estimating work performed by a worker based on speech,
at least one computer having a processor and a memory coupled to the processor;
The processor
converting the voice data of the utterances uttered by the worker into text, generating time-series data of the utterances in which a plurality of the texts are arranged in chronological order, storing the time-series data of the utterances in the memory;
utterance intervals between utterances are calculated from the plurality of speech data whose time series are continuous, time-series data of utterance intervals are generated by arranging the utterance intervals in chronological order, and time-series data of the utterance intervals are stored in the memory. store in
A computer system for estimating the contents and results of work performed by said worker based on said time-series data of said utterances and said time-series data of said utterance intervals, and outputting said result of said estimation. A computer system according to claim 1,
The processor
calculating sound pressure from the speech data, generating time-series data of sound pressure in which the sound pressure corresponding to each of the plurality of texts included in the time-series data of the utterance is arranged in time-series order; Store the time series data of in the memory,
A computer system for estimating the contents and results of work performed by said worker based on said time-series data of said utterances, said time-series data of said intervals of speech, and said time-series data of said sound pressures. A computer system according to claim 1,
The processor
Time-series data of utterances for analysis, time-series data of utterance intervals for analysis associated with the time-series data of utterances for analysis, and time-series data of utterances for analysis for each work type accepts time-series data of labels indicating the utterance intentions of utterances
Receiving speech data, which is information on assumed speech for each type of work,
Statistical analysis is performed using the time-series data of utterances for analysis, the time-series data of utterance intervals for analysis, and the time-series data of labels, and an utterance pattern indicating transition of utterance intention and the utterance pattern and time-series data of the utterance intervals of the utterance pattern in which the utterance intervals between utterances corresponding to the utterance intention in the utterance pattern are arranged in chronological order, and the utterance pattern and the occurrence probability of the utterance pattern are calculated. , and storing the time-series data of the utterance intervals of the utterance pattern in the memory,
generating utterance time-series data for learning using the utterance pattern, the probability of occurrence of the utterance pattern, and the utterance data, and storing the time-series data of the utterance for learning in the memory;
generating time-series data of utterance intervals for learning using the time-series data of utterance intervals of the utterance pattern, storing the time-series data of utterance intervals for learning in the memory;
generating time-series data of correct labels indicating a combination of correct work contents and results based on the utterance pattern, storing the time-series data of correct labels in the memory;
generating learning data composed of time-series data of utterances for learning, time-series data of utterance intervals for learning, and time-series data of correct labels, storing the learning data in the memory;
executing a learning process for generating a model that outputs the content and results of the work corresponding to the time-series data of the correct label using the time-series data of the utterance and the time-series data of the utterance interval as inputs; A computer system, wherein model information is stored in the memory. A computer system according to claim 1,
The interval between the utterances is at least one of a time interval between a reference utterance and an utterance one chronologically before the reference utterance, and a time interval between words included in the utterance. computer system. A method for estimating work performed by a worker based on utterances, executed by a computer system, comprising:
The computer system includes at least one computer having a processor and a memory connected to the processor;
The method of estimating the work includes:
The processor converts voice data of utterances uttered by the worker into text, generates time-series data of utterances in which a plurality of texts are arranged in chronological order, and stores the time-series data of utterances in the memory. a first step of storing;
The processor calculates utterance intervals between the utterances from a plurality of the speech data whose time series are continuous, generates time-series data of the utterance intervals by arranging the utterance intervals in chronological order, and generates the time-series data of the utterance intervals. a second step of storing data in said memory;
a third step in which the processor estimates the content and result of the work performed by the worker based on the time-series data of the utterance and the time-series data of the utterance interval, and outputs the result of the estimation; A work estimation method comprising: A work estimation method according to claim 5,
In the first step, the processor calculates a sound pressure from the speech data, and arranges the sound pressure corresponding to each of the plurality of texts included in the time-series data of the utterance in chronological order. generating time-series data of and storing the time-series data of the sound pressure in the memory;
In the third step, the processor determines the content and result of the work performed by the worker based on the time-series data of the speech, the time-series data of the speech interval, and the time-series data of the sound pressure. A method of estimating work, comprising a step of estimating. A work estimation method according to claim 5,
The processor provides, for each work type, time-series data of utterances for analysis, time-series data of utterance intervals for analysis associated with the time-series data of utterances for analysis, and time of the utterances for analysis. a step of receiving time series data of labels indicating utterance intentions of utterances included in the series data;
a step in which the processor receives utterance data, which is information on an assumed utterance for each work type;
The processor performs statistical analysis using the time-series data of utterances for analysis, the time-series data of utterance intervals for analysis, and the time-series data of labels, and an utterance pattern indicating transition of utterance intention. and calculating the appearance probability of the utterance pattern and time-series data of utterance intervals of the utterance pattern in which utterance intervals between utterances corresponding to the utterance intention in the utterance pattern are arranged in chronological order, and calculating the utterance pattern, the utterance, a step of storing in the memory time-series data of the pattern appearance probability and the utterance interval of the utterance pattern;
the processor generating utterance time-series data for learning using the utterance pattern, the appearance probability of the utterance pattern, and the utterance data, and storing the time-series data of the utterance for learning in the memory; and,
the processor generating time-series data of utterance intervals for learning using the time-series data of utterance intervals of the utterance pattern, and storing the time-series data of utterance intervals for learning in the memory;
a step in which the processor generates time-series data of correct labels indicating a combination of correct work contents and results based on the utterance pattern, and stores the time-series data of correct labels in the memory;
The processor generates learning data composed of time-series data of utterances for learning, time-series data of utterance intervals for learning, and time-series data of correct labels, and stores the learning data in the memory. storing;
The processor receives the time-series data of the utterance and the time-series data of the utterance interval as input, and performs learning processing for generating a model that outputs the content and result of the work corresponding to the time-series data of the correct label. executing and storing information of said model in said memory. A work estimation method according to claim 5,
The interval between the utterances is at least one of a time interval between a reference utterance and an utterance one chronologically before the reference utterance, and a time interval between words included in the utterance. How to estimate the work to be

Images