KR20230030502A - Method for generating conversaion information using examplar-based generation model and apparatus for the same - Google Patents

Abstract

A method for normalizing an embedding of a neural network model according to various embodiments of the present disclosure may comprise: a step of acquiring a plurality of feature vectors; a step of acquiring first embedding information by individually embedding the plurality of feature vectors; a step of acquiring normalized values of the first embedding information through a first parameter and a second parameter; and a step of outputting a first result value by applying normalized values for each of the individual feature vector to a training model. Therefore, the present invention is capable of providing an effect that can generate various and interesting answers.

Description

Conversation information generation method and apparatus using an example-based generation model

Various embodiments of the present disclosure relate to a method and apparatus for generating conversation information using an example-based generation model.

Artificial intelligence (AI) is being used in various industries. Artificial intelligence, which operates in a manner similar to human thinking, can be used to extract features of an object that a target object to be a sample is to approach.

There is an open-domain conversation field as a field for creating a chatbot that can lead a natural conversation without a specific topic of conversation being determined. Artificial intelligence models used in the field of open-domain conversation are largely divided into a generation-based conversation model and a search-based conversation model.

The generation-based dialog model generates an appropriate response to the dialog context information by using extracted and input dialog context information based on a sequence to sequence architecture. In the search-based dialog model, a response set that can be used as a response is first defined in advance, and a response most suitable for input dialog context information is retrieved from the response set and returned as a response.

The generative-based dialogue model has the advantage of generating fluent responses that match the given dialogue context information based on the rich knowledge of the language model when used together with a large-scale language model. However, the generative-based dialogue model generates the most natural response To do this, they tend to produce benign and uninteresting responses that do not interrupt the flow of conversation.

On the other hand, the search-based dialogue model extracts heterogeneous responses that do not match the confirmed or entered dialogue context information, but has the advantage of providing various and interesting responses compared to the generation-based dialogue model. In addition, when used with a high-performance search library (eg, FAISS, etc.), it has the advantage of being able to quickly extract an appropriate response to conversation context information, compared to a generation-based conversation model.

Therefore, in order to take advantage of the natural response generation capability of the generation-based dialog model and the strengths of the two models that generate various and interesting responses of the search-based dialog model, responses extracted by the search-based dialog model for given dialog context information are generated. Example-based generation models have been proposed that provide an example of a base dialog model and generate a more natural response from a given example.

A method of using the correct answer of the generation-based dialog model when searching for example answers in the search-based dialog model has been proposed in a related study (Angela Fan, et al 2021. Augmenting transformers with knn based composite memory for dialog).

Previously proposed example-based generation models may ignore the answers extracted and provided by the search-based dialogue model without using them, or may provide the same answers as they are without considering the given examples. In addition, example-based generative models that have been proposed in the past tend to bias learning toward the above-described cases over time.

Accordingly, the method for generating dialogue information using an example-based generation model according to embodiments overcomes the above problems and proposes an example-based generation model that generates more diverse and fluent answers by appropriately using given examples.

A method of generating conversation information using an example-based generation model according to embodiments includes checking first context information; identifying a first response set corresponding to the first context information based on the first model; identifying a response subset selected from the first response set based on Gold response information corresponding to the first context information; and learning the second model based on the first context information and the response subset. may include at least one of them.

Also, a response subset according to embodiments may be selected from candidate responses identified based on the Gold response information and the clustering algorithm.

Also, the response subset according to embodiments may be selected by excluding at least one answer corresponding to a specific range from a value corresponding to the Gold response information in an embedding space among the candidate responses.

Furthermore, the dialog model training method according to embodiments may further include setting weight information based on the first context information for each response included in the response subset, and In the learning step, the second model may be learned based on the set weight information.

Furthermore, the weight information according to embodiments may be set based on a relevance score for each answer in the response subset, and the relevance score for the answer is a value corresponding to the first context information in an embedding space. And it can be calculated based on the value corresponding to the answer.

In addition, the second model according to embodiments may check second context information for conversation information obtained from a user, and provide gold response information for the second context information based on the second context information. there is.

Furthermore, the first context information according to embodiments may include at least one piece of conversation information obtained from a user.

Furthermore, the second model according to the embodiments may be learned by performing a backpropagation operation using a loss function calculated based on the weight information, and the weight information is calculated by normalizing relevance scores for each answer. It can be.

A method of generating conversation information using an example-based generation model according to embodiments learns to extract an answer suitable for context information of a conversation based on a combination of a search model 200 and a generation model 201, thereby providing information based on rich knowledge. It is possible to provide an effect of generating various interesting answers while generating fluent answers suitable for a given conversation context.

The electronic device according to the embodiments excludes answers within a range that is too close (or highly correlated) from the value corresponding to the gold response information in the embedding space among candidate example answers, so that the generation model 201 is suitable for the context and It helps you learn to come up with a variety of answers.

The electronic device according to the embodiments learns to generate an optimal answer by further considering example answers and a weight for each example answer, thereby appropriately reflecting the example answers to provide an answer that is appropriate for the context and not awkward, and at the same time It can be induced to generate fluent and creative answers, and users who use the dialog model can lead conversations that they do not get tired of.

1 is a schematic block diagram illustrating the configuration of an electronic device according to various embodiments of the present disclosure.
2 shows a part of a configuration diagram of an electronic device according to embodiments.
3 is a diagram illustrating an example of an overall result of generating answer information from context information by an electronic device according to embodiments.
4 illustrates examples of operations for learning a generation model unit to generate answer information from context information by an electronic device according to embodiments.
5 illustrates an example of an operation of a search model unit according to embodiments.
6 illustrates an example of a result of an operation for learning conversation information and answer information by a search model unit according to embodiments.
7 illustrates examples of operations of an electronic device according to embodiments.
8 to 11 show comparison of performance of an open-domain conversation model of an electronic device with other open-domain conversation models according to embodiments.

The terms used in the embodiments have been selected as general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like. In addition, in a specific case, there are also terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the corresponding description. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

In the entire specification, when a part is said to "include" a certain component, it means that it may further include other components, not excluding other components unless otherwise stated. In addition, terms such as "...unit" and "...module" described in the specification mean a unit that processes at least one function or operation, which is implemented as hardware or software, or a combination of hardware and software. It can be.

The expression of “at least one of a, b, and c” described throughout the specification means 'a alone', 'b alone', 'c alone', 'a and b', 'a and c', 'b and c' ', or 'all of a, b, and c'.

A “terminal” referred to below may be implemented as a computer or portable terminal capable of accessing a server or other terminals through a network. Here, the computer includes, for example, a laptop, desktop, laptop, etc. equipped with a web browser, and the portable terminal is, for example, a wireless communication device that ensures portability and mobility. , IMT (International Mobile Telecommunication), CDMA (Code Division Multiple Access), W-CDMA (W-Code Division Multiple Access), LTE (Long Term Evolution), etc. It may include a handheld-based wireless communication device.

Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily carry out the present disclosure. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

In describing the embodiments, descriptions of technical contents that are well known in the technical field to which the present disclosure belongs and are not directly related to the present disclosure will be omitted. This is to more clearly convey the gist of the present disclosure without obscuring it by omitting unnecessary description.

For the same reason, in the accompanying drawings, some components are exaggerated, omitted, or schematically illustrated. Also, the size of each component does not entirely reflect the actual size. In each figure, the same reference number is assigned to the same or corresponding component.

Advantages and features of the present disclosure, and methods for achieving them, will become clear with reference to embodiments described below in detail in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms, only the present embodiments make the disclosure of the present disclosure complete, and the common knowledge in the art to which the present disclosure belongs It is provided to fully inform the holder of the scope of the invention, and the present disclosure is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

At this time, it will be understood that each block of the process flow chart diagrams and combinations of the flow chart diagrams can be performed by computer program instructions. These computer program instructions may be embodied in a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment, so that the instructions executed by the processor of the computer or other programmable data processing equipment are described in the flowchart block(s). It creates means to perform functions. These computer program instructions may also be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular way, such that the computer usable or computer readable memory The instructions stored in are also capable of producing an article of manufacture containing instruction means that perform the functions described in the flowchart block(s). The computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to generate computer or other programmable data processing equipment. Instructions for performing processing equipment may also provide steps for performing the functions described in the flowchart block(s).

Additionally, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s). It should also be noted that in some alternative implementations it is possible for the functions mentioned in the blocks to occur out of order. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in reverse order depending on their function.

Artificial intelligence (AI) can be a type of computer program that mimics human intelligence through a series of logical algorithms that think, learn, and judge like humans. So-called artificial intelligence can process complex calculations in a processor corresponding to the human brain through a neural network resembling the human nervous system. In this specification, a process of normalizing and modeling features through machine learning and other learning, which may be included in deep learning, will be described. In this specification, the terms of machine learning and machine learning may be used interchangeably.

A neural network may refer to a network modeling an operating principle of neurons, which are basic units of the human nervous system, and a connection relationship between neurons. A neural network may be a data processing system in which individual nodes or processing elements are connected in a layer form. A neural network may include a plurality of layers, and each layer may include a plurality of neurons. In addition, the neural network may include synapses corresponding to nerve stimulators capable of transmitting data between neurons. In this specification, the terms layer and layer may be used interchangeably.

Specifically, a neural network may generally mean a data processing model having an ability to solve a given problem or a problem with a variable by changing synaptic coupling strength through repetitive learning of artificial neurons. In this specification, the terms neural network and artificial neural network may be used interchangeably.

A neural network may be trained using training data. Specifically, training may include a process of determining parameters of a neural network using feature data to achieve a purpose such as classification, regression analysis, or clustering of input data. More specifically, a weight or a bias may be a factor determining the parameter.

The neural network may train input data to classify or cluster according to a pattern, and the trained neural network may be referred to as a trained model. Specifically, training methods may be classified into supervised learning, unsupervised learning, semi-supervised learning, and reinforced learning. More specifically, supervised learning may be a method of machine learning for inferring a function from training data. Outputting continuous result values from functions inferred through machine learning may be regression analysis, and outputting result values by predicting a class of input data may be classification.

In supervised learning, a label for training data may be given, and the label may include a meaningful result value to be inferred by a neural network. Specifically, the resulting value to be inferred by the neural network may be labeling data. More specifically, training data and labeling data corresponding to the training data may be configured as one training set, and the neural network may obtain input values and result values in the form of the training set.

The training data may include a plurality of feature vectors, and the neural network may infer the training data, label individual feature vectors, and output labeling data as result values. The neural network may infer a function for correlation between each data through training data and labeling data. In addition, parameters for individual vectors can be optimized through feedback on the function inferred from the neural network.

An example-based generative model for open-domain conversation and a method for improving the example-based generative model are described.

The example-based generative model generates responses based on example answers extracted by the search model unit, and uses a generative model and a retrieval model. An electronic device according to embodiments described in this specification includes an example-based generation model in which a generation model and a search model are connected or combined. On the other hand, existing example-based generative models often ignore retrieved samples or generate answers that are overfitted to retrieved samples while generating answers. Accordingly, the electronic device according to embodiments includes a Connecting Retriever and Generator (CORGE) that connects a search model and a generation model, and the electronic device according to embodiments includes some or all operations for learning an example-based generation model. can do everything In the step of learning the example-based generation model, the electronic device according to embodiments may use a query for selecting an example answer using gold response information as well as conversation context information. After that, the electronic device according to embodiments may exclude example answers that are too similar to Gold response information in order to alleviate the above-described overfitting problem. Since some of the remaining example answers may not be related to a given context, the electronic device according to embodiments may train a generating model by additionally utilizing a relevance score between context information and example answers.

Meanwhile, gold response information according to embodiments may mean, for example, optimal response information for a specific context, appropriate response information, or preset response information. For example, It may be response information set to be supervised learning within training data.

Hereinafter, the correspondence between the terms used in this specification and terms (English terms) for papers related to this specification may be as follows.

Related Paper Terminology Terminology in this specification Exemplar-based generative model Example-based generative model Human evaluation human evaluation Retriever, retrieval model Search part, search model part, search model Generator, generation model generation unit, generation model unit, generation model One-to-many problem One-to-many problem Exempler example answer, example Response answer CORGE Electronic device according to embodiments, learning method according to embodiments, etc. Gold response Gold Response, Gold Response Information Appropriateness compatibility Informativeness informational Knowledge-grounded generation model knowledge-based generative models Context context, context information Relevance score relevance score Normalized relevance score Normalized association score Jaccard filter Jacquard filter Jaccard similarity Jackard Similarity Over-fitting overfitting

1 is a schematic block diagram illustrating the configuration of an electronic device according to various embodiments of the present disclosure.

An electronic device may include a device including a neural network. The electronic device is a device capable of performing machine learning using training data, and may include a device capable of performing learning using a model composed of a neural network. For example, an electronic device may be configured to receive, classify, store, and output data to be used for data mining, data analysis, intelligent decision making, and machine learning algorithms.

An electronic device may include various devices for training a neural network. For example, an electronic device may be implemented as a plurality of server sets, a cloud server, or a combination thereof. Specifically, the electronic device may obtain result values through data analysis or training through distributed processing.

Referring to FIG. 1 , an electronic device may include a processor 110, an input/output module 120, and a memory 130 as components. Components of the electronic device shown in FIG. 1 are not limited thereto and may be added or replaced.

The processor 110 may control or predict the operation of the electronic device through data analysis and machine learning algorithms. The processor 110 may request, search, receive, or utilize data to be trained, and may control the electronic device to execute a desired operation learned through training. The processor 110 may include, for example, a learning processor according to embodiments that requests, retrieves, receives, or utilizes learning data, pre-processes them, or performs learning using them.

The processor 110 may be configured to derive and sense a result value for an input value based on a user's input or natural language input. Processor 110 may include, for example, a learning processor according to embodiments configured to collect data for processing and storage. Data collection may include sensing data through a sensor, extracting data stored in the memory 130 , or receiving data from an external device through the input/output unit 120 .

The processor 110 may convert the operation history of the electronic device into data and store it in the memory 130 . The processor 110 may obtain the best result value for performing a specific operation based on the stored operation history data and the trained model.

When a specific operation is performed, the processor 110 may analyze a history according to the execution of the specific operation through data analysis and a machine learning algorithm. Specifically, the processor 110 may update previously trained data based on the analyzed history. That is, the processor 110 may improve the accuracy of data analysis and machine learning algorithms based on the updated data.

For example, the processor 110 according to embodiments or a learning processor included in the processor 110 may train a neural network using training data or a training set. For example, the processor 110 or a learning processor included in the processor 110 may train a neural network through data obtained by preprocessing an acquired input value. For another example, the processor 110 or a learning processor included in the processor 110 may train a neural network through preprocessing data stored in the memory 130 . Specifically, the processor 110 or a learning processor included in the processor 110 may determine an optimization model of the neural network and parameters used for optimization by repeatedly training the neural network using various training methods.

The input/output unit 120 may perform a function of transmitting data stored in the memory 130 of the electronic device or data processed by the processor 110 to another device or receiving data from another device to the electronic device. The input/output unit 120 may include, for example, a receiving unit, a transmitting unit, or a transceiver that receives data from another electronic device or transmits data to another electronic device. Furthermore, the input/output unit 120 according to embodiments includes one or more input/output modules for receiving or outputting electronic signals or data with components or devices physically or logically connected to the electronic device. can do.

The memory 130 may store a model trained in the processor 110 or the neural network. For example, the memory 130 may separately store a trained model or a model under training. Specifically, the memory 130 may store models of a process in which a neural network is trained and store a trained model according to a training history.

For example, the memory 130 according to embodiments may include a model storage unit and/or a database.

For example, the model storage unit may store a model (eg, a neural network model, etc.) being trained or already trained through the processor 110 . Also, the model storage unit may store a model in which the trained model is updated.

For example, the database may store input data that is an input value, training data for model training, model training history data, and the like. The input data stored in the database may be processed data suitable for model training and raw data that are not processed.

Checking, by the processor 110 according to embodiments, first context information; identifying a first response set corresponding to the first context information based on a first model; identifying a response subset selected from the first response set based on Gold response information corresponding to the first context information; and learning the second model based on the first context information and the response subset. can be performed.

This specification addresses a common shortcoming of example-based generative models used in open-domain conversations (e.g., (Wu et al., 2019; Cai et al., 2019; Gupta et al., 2020, etc.)). We propose a simple training method for example-based generative models, a retrieval model (e.g. Humeau et al., 2019; Mazarι et al., 2019; Mazarι et al., 2018) and generative models (Adiwardana et al., 2020; Roller et al., 2020; Zhang et al., 2020; Brown et al., 2020) into one framework. First, a retriever (search model) searches for an example using a given context as a query, and a generator (generation model) generates a response based on the given context and the example. Example-based generative models generate more specific responses than generative models, and more fluent responses than search models (Weston et al., 2018).

Despite the success of example-based generative models, these conventional models suffer from two drawbacks. As shown in Fig. 3 below, primitive example-based generative models (e.g., Weston et al., 2018; Cai et al., 2018) tend to ignore examples entirely, and vanilla generative models. model) appears to be similar. This is because the example retrieved from the presented contexts in the training phase is significantly different from the gold response (or gold response information), that is, a one-to-many problem (Li et al. , 2016). To alleviate this problem, example-based generative models may utilize a Gold response (Roller et al., 2020) or a perturbed gold response as an example within the training phase. However, these training methods make the generation unit overly dependent on the retrieved examples, and may improperly use the tokens provided by the generation model (eg, as in the embodiment of (b) in FIG. 3 below). These two disadvantages can adversely affect the quality of the generated response.

The present specification proposes a model that can mitigate these disadvantages by providing semantically related but appropriately distant examples from Gold response (Gold response information) information in the process of training example-based generative models. The model proposed in this specification is CORGE (Connecdting Retreiver and Generator), and as one training method, it proposes a method of selecting appropriate examples. First, CORGE according to embodiments may utilize the Gold response to select examples that are similar but not a priori identical to the Gold response. However, the selected examples may become irrelevant with the given context, in which case they may rely entirely on the Gold response. Accordingly, CORGE can calculate an association score with the context and the example. Then, CORGE assigns weight to similar examples using the score and penalizes examples that are similar to the given context information. CORGE is basically applicable to instance-based generative models and improves the quality of generated responses in terms of relevance and informativeness.

In other words, existing example-based generative models tend to over-rely on or ignore exemplars extracted and selected from the search model. That is, existing example-based generation models fail to reduce the frequency of generating response information that is overfitted to example answers extracted and selected from a search model or generating responses unrelated to example answers extracted and selected.

On the other hand, the model proposed in this specification (eg, a model based on the combination of 200 and 201 in FIG. 2) may have an improved configuration of the existing search model (eg, 200 in FIG. 2), and extract and re-verifying the example answers by removing example answers that may generate overfit response information from among the selected example answers or by assigning weights based on the relevance scores of the example answers. there is. With this configuration, the model proposed in this specification (for example, the model according to the combination of 200 and 201 in FIG. 2) avoids generating an overfitted response to the example answers extracted and selected from the search model, thereby generating a model It is possible to maximize the learning efficiency of negatives, and it provides an effect of helping natural conversations continue by reducing the frequency of generating responses unrelated to the extracted and selected example answers.

Accordingly, this specification describes the following contributions or suggestions, and proposes the construction of improved embodiments of example-based generative models for making the following contributions or suggestions (e.g., FIG. 2 Combination of 200 and 201 of).

1) This specification shows that existing example-based generative models ignore examples or generate responses that are overfitted to examples.

2) CORGE, a training method proposed in this specification, alleviates the above-mentioned disadvantages by selecting appropriate examples and applying weights to examples of relevance scores evaluated by a search model.

3) Human evaluation results show that CORGE dramatically improves the performance of example-based generative models in terms of relevance and informativeness.

Regarding the CORGE training method proposed in this specification, an example-based generative model will be described.

While generative models have shown successful performance in open-domain conversations, generative models are well-known for lacking information and presenting bland responses (Li et al., 2016; Liu et al., 2016; Serban et al., 2017; Li et al., 2019; Holtzman et al., 2019; Welleck et al., 2019). Instance-based generative models overcome the above-mentioned problems borne by generative models. Wu et al. (2019) introduced an example-based generative model for open domain conversations. This model retrieves a context-model pair conditioned according to the input context and encodes the lexical difference between the input context and the retrieved context into an edit vector. A response is generated by feeding the example and the edit vector. Weston et al. (2018); Roller et al. (2020) also retrieves examples using the presented context as a query, concatenates the examples and contexts, and feeds the combined examples to the generator to generate a final response for the open-domain conversation. Cai et al. (2018, 2019) use the masked example to remove extraneous information from the example and inform the generator to generate a response. Gupta et al. 2020 adjusts the generation unit with the extracted semantic frame of retrieved examples and statements. In this specification, we do not consider this model as a baseline because it requires an additional semantic frame extractor and can be complemented with the proposed training method.

In summary, the electronic device according to the embodiments may perform the above-described training method performed by CORGE. Different from existing example-based generation models, the electronic device according to the embodiments selects example answers that are similar but have a certain distance, and calculates relevance scores for the selected example answers to weight them. By giving , it is possible to ignore the selected example or not to generate an overfitting response to the selected example. A detailed configuration of an electronic device according to embodiments is described in FIG. 2 , a process of generating a correct response by an electronic device according to embodiments is described in FIGS. 3 to 7 , and FIGS. 8 to 11 are illustrated in FIGS. shows the advanced effect on the performance of the electronic device according to

2 shows a part of a configuration diagram of an electronic device according to embodiments.

Specifically, FIG. 2 shows a configuration or a set of instructions for an electronic device to learn an artificial intelligence model for open-domain conversation according to embodiments. An electronic device according to embodiments may represent part or all of an example-based generation model for open open-domain conversations. In the electronic device according to embodiments, the search model unit 200 receives context information about a conversation and gold response information corresponding to the corresponding context information, and learns them. An electronic device according to embodiments may include an artificial intelligence model that learns the above information, receives conversation information (or context information), and predicts or generates response information for the corresponding conversation information.

In this specification, the following notation will be used to explain the structure and operation of the example-based generative model, and problems of existing models will be described.

는 다이얼로그 데이터세트(dialogue dataset)를 나타내며, 컨텍스트 정보 c 및 답변 정보 r를 이루는 n개의 짝들로 이루어진다. 예시-기반 생성 모델들은 두 개의 구성요소를 포함할 수 있다: 하나는 검색부 (

, Retriever), 및 생성부 (

, generator). 주어진 컨텍스트 정보

에 대하여, 검색부는 기-정의된 답변 세트

에 있는 예시

의 연관성 점수

의 최-고점(top-scoring)의 예시를 확인한다. 상기 생성부는 예시 z를 활용하여 컨텍스트 정보

에 대한 답변의 확률

를 계산한다.

represents a dialog dataset, and consists of n pairs constituting context information c and answer information r. Example-based generative models can include two components: a search unit (

, Retriever), and generating unit (

, generator). given context information

For , the search unit pre-defined set of answers

example in

relevance score of

Check out examples of top-scoring of . The generation unit utilizes example z to provide context information

probability of answering

Calculate

Here, the search unit may be referred to as, for example, a retrieval model, and the generation unit may be referred to as, for example, a generative model.

For example, one example answer may be expressed (or embedded) as a list or vector of data having a plurality of dimensions, and thus one example answer may be expressed in an embedding space ( It can be expressed as one or more positions in embedding space. Similarly, context information according to embodiments may also be expressed (or embedded) as a list or vector of data having a plurality of dimensions, and Can be expressed in one or more locations.

)는 적어도 두 예시 답변들 간의 연관성, 또는 컨텍스트 정보와 특정 예시 답변 간의 연관성을 나타낼 수 있는 지표를 나타낼 수 있다. 예를 들어, 연관성 점수는 예를 들어 임베딩 공간 상에 위치하는 두 개의 예시 답변들 (또는 하나의 예시 답변과 하나의 컨텍스트 정보) 간의 연관성을 나타낼 수 있다. Relevance score according to embodiments (relevance score,

) may indicate an index that may indicate a correlation between at least two example answers or a correlation between context information and a specific example answer. For example, the correlation score may represent a correlation between two example answers (or one example answer and one context information) located in the embedding space.

On the other hand, the concept of 'relevance' described in this specification is based on the value of the embedded vector of two example answers located on the embedding space, the distance between the two vectors, the degree of density, the degree of similarity (similarity), etc. It can include concepts that can mean.

은 임베딩 공간 상에 표현되는 복수의 예시 답변들을 확인할 수 있고 저장할 수 있는 검색 모델부를 의미할 수 있다.

는 실시예들에 따른 컨텍스트 정보를 나타낼 수 있으며, z는 실시예들에 따른 복수의 예시 답변들 중 하나를 의미할 수 있다.here,

may denote a search model unit capable of checking and storing a plurality of example answers expressed on the embedding space.

may indicate context information according to embodiments, and z may mean one of a plurality of example answers according to embodiments.

A method of calculating a correlation score according to embodiments may be, for example, as shown in FIG. 13 . Meanwhile, for example, referring to FIG. 2 , an electronic device according to embodiments may include a search model unit 200 ) and a generation model unit 201.

The generation model unit 201 according to embodiments may include an artificial intelligence model that outputs response information suitable for the context of a corresponding conversation upon receiving input of context information of the conversation. The artificial intelligence model included in the generation model unit 201 may include an artificial neural network model learned by training set data, and the training set data may include, for example, context information and Gold response information corresponding to the corresponding context information may be included.

Here, the training set data according to the embodiments is information on example answer(s) so that gold response information corresponding to appropriate context information can be output from context information on the conversation. may further include. That is, the generative model 201 according to the embodiments may train an artificial intelligence model by further utilizing information on example answer(s) as well as context information of conversation and gold response information.

Accordingly, the search model unit 200 according to embodiments may generate example answer(s) according to embodiments. The search model unit 200 may generate or select example answer(s) according to embodiments by receiving context information about a conversation and gold response information corresponding to the corresponding context information, Generated or selected example answers may be transmitted as training set data to the generated model 201 according to embodiments.

An electronic device according to embodiments learns to extract a response suitable for context information of a conversation based on an example-based generation model in which the search model 200 and the generation model 201 are combined. By doing so, it is possible to generate a fluent response suitable for a given conversational context based on rich knowledge and at the same time to provide an effect capable of generating various and interesting responses.

On the other hand, the example-based generation model including the conventional search model and the conventional generative model has the following disadvantages.

For example, a primitive example-based generative model (Weston et al., 2018), as shown in Roller et al., (2020), which discloses a conventional example-based generative model that includes a search model and a generative model. tend to ignore examples retrieved by the one-to-many problem in open-domain conversations during response generation (Li et al., 2016). Since the search model extracts an example answer based on the context, even if both the retrieved example and the gold response are similar to the given context information, the example retrieved as shown in (a) of FIG. 3 is the gold response information (gold response, gold response). ) may be different from each other. Since retrieved examples do not help generate gold responses, conventional generative models are trained to ignore retrieved examples, and can generate responses using only the contextual information.

및 골드 응답

가 과도하게 유사하면 (예를 들어, 도 3의 (b) 실시예), 종래의 예시-기반 생성 모델은 상기 예시에 과도하게 의존하도록 학습될 수 있다. 이 경우 결과적으로, 생성 모델은 상기 예시의 토큰들을 직접적으로 복사함으로써 예시에 과도하게 적합된(over-fitted) 응답을 생성할 수 있다.Rather than using retrieved examples during model training, Roller et al. (2020) utilizes the Gold response, and Cai et al. (2019) use the perturbed gold response as an example. However, an example answer

and gold response

If is excessively similar (eg, embodiment (b) of FIG. 3 ), a conventional example-based generative model may be learned to rely excessively on the example. As a result in this case, the generative model may generate an over-fitted response to the example by directly copying the tokens of the example.

In summary, when learning based on an example-based generation model that combines the conventional search model and the conventional generative model described above, the generative model ignores given examples or gives examples ( ) may be used as is to derive a response. For example, if an example answer selected or confirmed by a search model has very little relevance to actual context information, the generative model cannot perform correct learning by ignoring the given example. In addition, if the example answers selected or confirmed by the search model are highly related to actual context information, the generative model only uses and outputs the given example as it is and cannot perform correct learning.

Specific problems of the above-described example-based generative model including the conventional search model and the conventional generative model will be described in detail with reference to FIG. 3 .

In order to overcome the problems of the above-described conventional search model and example-based generation model including the conventional generation model, the search model unit 200 according to the embodiments includes a response storage unit 200a, a candidate example answer confirmation unit 200b, an example answer selection unit 200c, and a weight calculation unit 200d may further include at least one.

The response storage unit 200a may include information on a plurality of responses. The response storage unit 200a may store contents of a plurality of responses, and may store a position or value of each response in an embedding space.

The candidate example answer checking unit 200b may check one or more candidate example answers among a plurality of responses stored in the response storage unit 200a. The candidate example answer checking unit 200b may check candidate example answers using gold response information received as a training set.

Specifically, the candidate example answer(s) may refer to responses included within a first range from a value (or location) of a suitable response on an embedding space. Here, the first range for identifying the candidate example answer(s) may be determined based on, for example, a clustering algorithm. For example, the candidate example answer(s) is based on the value (or location) of the appropriate response and a clustering algorithm (eg, k-means algorithm, k-Nearest Neighbor (kNN) algorithm, etc.) on the embedding space, It may mean the response(s) selected from the responses of.

The example answer selector 200c according to embodiments may select one or more example answer(s) from the candidate example answer(s) checked and selected by the candidate example answer checker 200b. In order for the generative model 201 according to the embodiments to induce high-efficiency learning due to the example answers selected from the search model 200, the search model 200 has too high correlation with the context information according to the embodiments. Or, at least one of the example answer(s) and the response(s) with high or low relevance may be removed from the gold response information. Accordingly, the example answer selector 200c may select example answers suitable for learning from among the candidate example answers checked and selected by the candidate example answer checker 200b.

The example answer(s) selected by the example answer selection unit 200c is, for example, a value within the second range that is too close (or highly correlated) to a value corresponding to the gold response information in the embedding space among the candidate example answer(s). It can also be selected by excluding (response). For example, the second range may mean a range having similarity equal to or greater than a certain threshold based on Jaccard similarity.

The electronic device according to the embodiments excludes responses within a range that is too close (or highly correlated) from the value corresponding to the gold response information in the embedding space among candidate example answers, so that the generation model 201 is appropriate for the context and It helps you learn to derive a variety of responses.

The search model 200 according to embodiments may provide example answers (or information on example answers) selected by the example answer selector 200c to the generation model 201 according to embodiments.

Meanwhile, the generation model 201 according to the embodiments may be trained using not only selected example answers but also weight value(s) related to example answers. The weight value(s) associated with example answers may indicate, for example, the extent to which example answers according to embodiments are associated with context information and/or gold response information according to embodiments. For example, the weight value(s) may mean a relevance score or a normalized relevance score for each example answer.

Accordingly, the search model unit 200 according to embodiments further includes a weight calculation unit 200d that calculates or derives weight value(s) for each example answer from the example answers selected by the example answer selection unit 200c. can include The weight calculation unit 200d calculates a relevance score or a normalized relevance score for each example answer using selected example answers according to embodiments and/or context information according to embodiments, thereby calculating a relevance score for each response. weights can be calculated.

The electronic device according to the embodiments learns to generate an optimal answer by further considering example answers and a weight for each example answer, thereby appropriately reflecting example answers to provide a response that is appropriate for the context and is not awkward at the same time. It can be induced to generate fluent and creative responses, and users who use the dialog model can lead conversations that they do not get tired of.

The generative model 201 according to the embodiments may learn the artificial neural network model included in the generative model 201 using the above-described selected example answers and weight information for each example answer.

Specifically, the generative model 201 according to the embodiments may perform forward propagation of the artificial neural network model based on selected example answers (and/or weight information for each example answer) and responds can create The generation model 201 according to embodiments may provide generated response information to a user.

In addition, the generation model 201 according to the embodiments may generate a loss function using the above-described gold response information and selected example answers (and/or weight information for each example answer), The artificial neural network model may be updated or learned by performing back propagation of the artificial neural network model included in the generation model 201 using the generated loss function.

A loss function according to embodiments may mean, for example, a function or value required to perform a backpropagation operation performed to learn a neural network. Accordingly, the loss function according to the embodiments may be, for example, similarity scores (or normalized values) for each answer calculated based on each example answer and context information for each example answer selected by the search model unit 200. It can be determined based on the similarity score).

A method of generating conversation information using an example-based generation model according to embodiments learns to extract a response suitable for context information of a conversation based on a combination of a search model 200 and a generation model 201, thereby providing rich knowledge based It is possible to provide an effect capable of generating various interesting responses while generating a fluent response suitable for a given conversation context.

Meanwhile, the search model unit 200 according to embodiments may be variously called a search unit, a retriever, and the like. Hereinafter, examples of operations of the example answer selector 200c and examples of operations of the weight calculation unit 200d according to embodiments will be described in detail.

3 is a diagram illustrating an example of overall results of generating response information from context information by an electronic device according to embodiments.

The operations shown in FIG. 3 are the search model unit 200 and the generation model unit 201 according to the embodiments learned based on the operations shown in FIG. 2 (eg, the learned artificial neural network according to the embodiments) model, etc.), an example of an operation of generating response information from conversation information acquired from a user is shown.

Specifically, FIG. 3 shows that after the electronic device according to embodiments checks context information from conversation information (300), selects example answer(s) based on the context information (301), and selects the selected example answer(s). It shows an example of an operation of generating 302 response information corresponding to conversation information using .

Referring to FIG. 3 , an electronic device according to embodiments may generate or check (300) context information using conversation information (or given conversation information) input from a user. Context information includes one or more conversations received from the user (eg, "A: Do you ever feel like time is just going by way too fast?", etc.), and responses provided for each conversation (eg, "A: Do you ever feel like time is just going by way too fast?"). "B: OMG! Especially recently. A week seems like one day.").

Referring to FIG. 3 , an electronic device according to embodiments may search for and select example answer(s) by using the checked context information as a query (301). The operation of selecting the example answer(s) may be performed based on some or all of the operations of the search model 200 of FIG. 2 , for example. Referring to FIG. 3 , an electronic device according to embodiments generates a response to a conversation using the selected example answer(s) (302).

Meanwhile, in the process of learning training set data, the generation model unit according to the embodiments may be trained in various ways according to the characteristics of example answers provided from the search model unit, the contents of example answers, and other data provided. Looking at 302a to 302c of FIG. 3 , the generation model unit according to the embodiments may generate a response unrelated to the context information 300 and the selected example answer 301 as in the first example 302a, and the second example 302a. An excessively identical response to the selected example answer 301 may be generated, such as 302b.

When the example answer(s) selected and provided by the search model unit is considerably far from the gold response information (ie, when the difference between the values in the embedding space is large), the electronic device responds to the above-described first example 302a. Likewise, it can be erroneously trained to derive answers that are not related to conversation information. In addition, when the example answer(s) selected and provided by the search model unit is very similar to the gold response information (ie, the difference in values in the embedding space is extremely small), the electronic device uses the above-described second example 302a. ), it can be erroneously trained to derive gold response information as it is. That is, there is a so-called one-to-many problem in which there may be various example answers that can be selected from one context information, and a gap in similarity may occur even within the selected example answers. Therefore, the learning effect of the artificial neural network is reduced, and at the same time, the possibility of causing an overfitting problem is increased.

Accordingly, the electronic device according to the embodiments may select example answer(s) based on a specific method in order to generate response information so as to be fluent and guarantee continuity of conversation as in the third example 302c. .

Conversation information generation using an example-based generation model according to embodiments Based on rich knowledge, a fluent response suitable for a given conversation context can be generated, and at the same time, various interesting responses can be generated.

4 illustrates examples of operations for learning a generation model unit to generate response information from context information by an electronic device according to embodiments.

Specifically, FIG. 4 illustrates part or all of an operation of learning example answer(s) in order to generate response information from conversation information by an electronic device according to embodiments. Operations shown in FIG. 4 may be performed by the search model unit 200 and the generation model unit 201 in the electronic device of FIG. 2 .

Context information according to embodiments may correspond to being embedded with a specific value on the embedding space 400 . In addition, a plurality of responses according to embodiments (eg, responses stored in the response storage unit 200a of FIG. 2 ) may also be embedded with a specific value on the embedding space 400 . For example, the response storage unit 200a of FIG. 2 may store specific values embedded in the embedding space 400 for each response.

An electronic device according to embodiments may search for response(s) within a specific range 400a from a value on an embedding space corresponding to context information among a plurality of responses. The electronic device according to embodiments may select example answer(s) 403a and 403b to be provided to the generation model unit from among the response(s) within the specific range 400a. Meanwhile, the electronic device according to embodiments may check gold response information 402 included in training set data.

Meanwhile, in the process of learning the training set data, looking at 403a to 403b of FIG. 4 , the generation model unit according to the embodiments may select an example answer unrelated to the gold response information 402 as in the first example 403a. there is. Similarly, a response that is excessively identical to the selected gold response information 402 may be selected as in the second example 403b.

The generation model unit 404 of the electronic device according to the embodiments may, in order to learn the artificial neural network model in the generation model unit 404, use gold response information and selected example answers (and/or weight information of each answer) A loss function may be calculated based on the loss function, and the artificial neural network in the generation model unit 404 may be trained by performing a backpropagation operation of the artificial neural network in the generation model unit 404 based on the loss function.

However, when the first example 403a is selected as an example answer, the generation model unit 404 may ignore the corresponding example answer due to a large difference from gold response information. On the other hand, when the second example 403b is selected as an example answer, the generation model unit 404 excessively considers the corresponding example answer with a very small difference from the gold response information, and thus an overfitting phenomenon may occur. Accordingly, the electronic device according to the embodiments may select example answer(s) based on a specific method in order to generate response information that is fluent and ensures continuity of conversation.

5 illustrates an example of an operation of a search model unit according to embodiments.

)을 선택하는 방법을 나타낸 일 실시예이다. 5 is a search model unit according to embodiments according to FIGS. 3 to 4, example answer(s) from a plurality of responses (

) is an embodiment showing a method of selecting.

Referring to FIG. 5 , the search model unit according to embodiments includes an operation of identifying candidate example answer(s) from gold response information (5A), and an operation of selecting example answer(s) from candidate example answer(s) ( At least one of 5B) and an operation 5C of calculating a weight for each selected example answer may be performed. In FIG. 5, 5A is, for example, by the candidate example answer checker 200b of FIG. 2, 5B is by, for example, the example answer selector 200c of FIG. 2, and 5C is, for example, weight calculation in FIG. It may be performed by unit 200d.

Referring to 5A of FIG. 5 , the search model unit according to the embodiments may check context information 501, and from the context information 501, within the association range 501a on the embedding space 500 or gold response information ( From 502 , it is possible to check candidate example answer(s) 503 existing within the first range 502a on the embedding space 500 . The first range 502a according to embodiments may represent a range determined based on a clustering algorithm (eg, k-means algorithm, k-Nearest Neighbor (kNN) algorithm, etc.).

The electronic device according to embodiments may select candidate example answers based on a clustering algorithm, etc., and ultimately prevent the selected example answer from being ignored or an underfitting phenomenon from occurring. In addition, with this operation, unnecessary delay in the learning process can be reduced because the selected example answer is not ignored.

Referring to 5B of FIG. 5 , the search model unit according to embodiments may exclude answer(s) included in the second range 504 from among selected candidate example answer(s) 503 . Here, the second range 504 may be a range determined by a user or a system. For example, the second range 504 may mean a Jaccard Filter Boundary on an embedding space.

The electronic device according to embodiments may prevent learning from answers that are excessively similar to gold response information by excluding these answer(s), ultimately preventing an overfitting phenomenon.

Meanwhile, the answers selected by 5A and 5B have a high degree of association with gold response information, but there may be a difference in degree of association with context information according to embodiments. For example, within the answers selected by 5A and 5B, the correlation with context information is high, but the correlation with gold response information may be low, depending on the characteristics of the clustering algorithm (eg, kNE) performed in 5A, and the context information Although correlation with is low, correlation with gold response information may be high. The electronic device according to the embodiments needs to learn the generation model unit considering the correlation between each example answer and the context information. In addition, if the search model unit performs only the above-described operations 5A and 5B, the selected example answer(s) may include answers entirely dependent on gold response information, which may reduce learning efficiency.

Accordingly, according to 5C of FIG. 5 , the search model unit according to the embodiments may calculate a degree of similarity (eg, weight) between each answer and context information. A weight according to embodiments may indicate similarity between one answer and context information, and similarity may be calculated based on, for example, a difference between values in an embedding space. The search model unit according to embodiments may calculate weight values as many as the number of example answers using context information, and may map each calculated weight to each example answer and provide the calculated weight values to the generation model unit according to embodiments.

) 는, 선택된 예시 답변들(exemplar

), 및 실시예들에 따른 컨텍스트 정보 (

)에 기반하여 각 예시 답변마다 계산 유사성 점수(relevance score,

)할 수 있다. 나아가, 일 실시예들에 따른 검색 모델부는 각 예시 답변에 대한 유사성 점수(

)에 소프트맥스(softmax) 함수를 적용하여 정규화된 유사성 점수 (

)를 계산할 수 있다. 그 후, 실시예들에 따른 전자 장치는 각 답변에 대한 정규화된 유사성 점수들을 이용하여 기존의 가능도(traditional likelihood)를 가중치 가능도(weighted likelihood)로 변환하고, 변환된 가중치 가능도로부터 생성된 손실 함수(loss function)을 최소화하도록 실시예들에 따른 생성 모델부를 학습시킬 수 있다. 실시예들에 따른 생성 모델부를 학습시키기 위하여, 변환된 가중치 가능도로부터 생성된 손실 함수(loss function, L)는 예를 들어, 아래와 같이 계산될 수 있다.Specifically, the search model unit according to an embodiment (

) is the selected example answers (exemplar

), and context information according to embodiments (

), calculated similarity score for each example answer (relevance score,

)can do. Furthermore, the search model unit according to an embodiment has a similarity score for each example answer (

) by applying the softmax function to the normalized similarity score (

) can be calculated. After that, the electronic device according to the embodiments converts the traditional likelihood into a weighted likelihood using normalized similarity scores for each answer, and generates a weighted likelihood generated from the converted weighted likelihood. A generative model unit according to embodiments may be trained to minimize a loss function. In order to train the generative model unit according to the embodiments, a loss function (L) generated from the transformed weight likelihood may be calculated as follows, for example.

[Equation 1]

)는, 계산된 손실 함수(L)을 이용하여 역전파 동작을 수행할 수 있다. 실시예들에 따른 역전파 동작을 수행하는 과정에서 계산되는 기울기 즉, 그래이디언트(gradient)는 예를 들어 다음과 같이 계산될 수 있다.Generation model unit according to the embodiments (

) may perform a backpropagation operation using the calculated loss function (L). A gradient, that is, a gradient calculated in a process of performing a backpropagation operation according to embodiments may be calculated as follows, for example.

[Equation 2]

)에 의해 스케일(scale)되었음을 증명하며, 선택된 예시 답변(들) z가 컨텍스트 정보 (

)와 연관성이 적은 경우 생성 모델부가 적게 업데이트(즉, 적게 변화하도록 학습)됨을 나타낸다.Equation 2 as an example is a similarity score in which the gradient of the generative model unit is normalized (

), and the selected example answer(s) z is the context information (

) and a small correlation indicates that the generative model part is updated less (that is, learns to change less).

Due to the above-described operations of the search model unit and the generation model unit, the electronic device according to embodiments may ignore inappropriate or irrelevant example answer(s) or may induce learning by considering them less. Also, with this configuration, the electronic device can learn a generating model to easily fetch tokens from example answers associated with gold response information.

6 illustrates an example of a result of an operation for learning conversation information and response information by a search model unit according to embodiments.

Specifically, FIG. 6 shows an example of an operation for learning conversation information and response information of the search model unit according to the embodiment shown in FIG. 5 . Referring to FIG. 6 , the search model unit according to embodiments may check context information 600 and gold response information 601 using conversation information.

' 는 실시예들에 따른 검색 모델부에 의해 계산된 각 응답에 대한 정규화된 연관성 점수를 나타낼 수 있다. 또한, 'Use?' 는 실시예들에 따른 전자 장치가 생성 모델부의 학습을 위하여 생성 모델부로 예시 답변을 제공할지 여부를 나타낸다.Referring to FIG. 6 , 602 may mean a candidate example answer retrieved from a search model unit using context information as a query among a plurality of responses according to embodiments. Referring to FIG. 6 , 603 may mean candidate example answer(s) extracted based on a clustering algorithm (eg, kNE) according to embodiments. In 602 and 603, 'Sim' may indicate a literal similarity between the gold response information and each candidate example answer, and '

' may indicate a normalized relevance score for each response calculated by the search model unit according to embodiments. Also, 'Use?' Indicates whether the electronic device according to the embodiments provides an example answer to the generative model unit for learning of the generative model unit.

7 illustrates examples of operations of an electronic device according to embodiments.

Some or all of the operations shown in FIG. 7 may be performed by, for example, the processor 110 of FIG. 1 or a learning processor included in the processor 110, and the search model 200 of FIG. ) and/or by the generative model 201.

Referring to FIG. 7 , an electronic device according to embodiments may check first context information (700). Referring to FIG. 7 , an electronic device according to embodiments may check a first response set corresponding to the first context information based on a first model (701). The first response set shown in FIG. 7 may mean example answers included in the association range 501a shown in FIG. 5 . The first context information according to embodiments may include at least one piece of conversation information obtained from a user.

Referring to FIG. 7 , the electronic device according to embodiments may check a response subset selected from the first response set based on appropriate response information corresponding to the first context information (702). The response subset shown in FIG. 7 may mean a candidate response set according to embodiments. For example, the response subset shown in FIG. 7 may represent answers other than the example answers included in the second range 504 among the example answers included in 502 of FIG. 5 .

Meanwhile, a response subset according to embodiments may be selected from candidate responses identified based on Gold response information and a clustering algorithm according to embodiments. Also, a response subset according to embodiments may be selected by excluding at least one answer corresponding to a specific range from a value corresponding to the Gold response information in an embedding space among candidate responses.

Referring to FIG. 7 , the electronic device according to embodiments may learn the second model based on the first context information and the response subset (703).

The electronic device according to embodiments may further include setting weight information for each response included in the response subset based on the first context information. Here, the electronic device according to embodiments may learn the second model based on set weight information.

Weight information according to embodiments may be set based on a relevance score for each answer in the response subset, and the relevance score for an answer corresponds to a value corresponding to the first context information and the answer in an embedding space. It can be calculated based on the value of

Meanwhile, the second model according to embodiments may check second context information for conversation information obtained from a user, and provide gold response information for the second context information based on the second context information.

The second model according to the embodiments may be learned by performing a backpropagation operation using a loss function calculated based on the weight information, and the weight information may be calculated by normalizing relevance scores for each answer. .

A method of generating conversation information using an example-based generation model according to embodiments learns to extract an answer suitable for context information of a conversation based on a combination of a search model 200 and a generation model 201, thereby providing information based on rich knowledge. It is possible to provide an effect of generating various interesting answers while generating fluent answers suitable for a given conversation context.

The electronic device according to the embodiments excludes answers within a range that is too close (or highly correlated) from the value corresponding to the gold response information in the embedding space among candidate example answers, so that the generation model 201 is suitable for the context and It helps you learn to come up with a variety of answers.

The electronic device according to the embodiments learns to generate an optimal answer by further considering example answers and a weight for each example answer, thereby appropriately reflecting the example answers to provide an answer that is appropriate for the context and not awkward, and at the same time It can be induced to generate fluent and creative answers, and users who use the dialog model can lead conversations that they do not get tired of.

8 shows an example of an operation of an electronic device according to embodiments.

Specifically, FIG. 8 illustrates an example of an operation of a conversational electronic device that performs an open-domain conversation according to embodiments. For example, the operation of FIG. 8 may represent an example of an operation for extracting answer information from conversation information input from an actual user, rather than a process of learning to extract a response from training data.

Referring to FIG. 8 , an electronic device according to embodiments may receive conversation information from a user. An electronic device according to embodiments may extract context information 8a from received conversation information. Context information 8a according to embodiments may be extracted by the processor 110 in the electronic device or a learning processor included in the processor 110, and the embodiments described above with reference to FIGS. 2 to 11 It may mean context information according to .

An electronic device according to embodiments may further include at least one of a search model unit 800 and a generation model unit 801, each of which includes the search model unit 800 and the generation model according to FIGS. 2 to 11 It may mean part 801.

An electronic device according to embodiments may receive context information 8a extracted from conversation information input from a user and generate an appropriate answer 8b for the corresponding conversation information. Accordingly, the electronic device may transmit the context information 8a to the search model unit 800 . The KNN unit 800b in the search model unit 800 according to embodiments stores context information 8a according to embodiments and a plurality of answers stored in the example answer storage unit 800a of the search model unit 800. Based on this, candidate example answers can be extracted. The example answer selector 800c in the search model unit 800 according to embodiments may select one or more of the extracted candidate example answers. The weight calculation unit 800d in the search model unit 800 according to embodiments may calculate weights (or calculate relevance scores) from one or more selected candidate example answers. The search model unit 800 according to embodiments may transmit the selected example answers to the generation model unit 801 according to embodiments. The generation model unit 801 according to the embodiments may receive selected example answers and context information 8a according to the embodiments, and transmit them to the artificial neural network model 801a included in the generation model unit 801. It can be input to the input layer, and response information 8b can be generated and checked from the output layer. The generation model unit 801 may output the answer information 8b checked from the output layer and provide it to the user.

The artificial neural network model 801a included in the generation model unit 801 performs a forward propagation operation 801b using the selected example answers received as input layers and the context information 8a according to the embodiments. can

On the other hand, the generation model 801 according to embodiments extracts response information 8b from a conversation input from a user in order to induce an immediate and real-time conversation, and then performs a backpropagation process (and/or Alternatively, an operation of calculating a loss function according to embodiments, etc.) may be omitted. On the other hand, if the backpropagation process (and/or the operation of calculating a loss function according to the embodiments, etc.) is omitted, the artificial neural network model 801a of the generation model 801 according to the embodiments is learned. Some embodiments of the electronic device described in this specification may be in that an immediate response may be extracted from a conversation input from a user instead of being lost. Therefore, only by describing an embodiment in which the back-propagation process (and/or the operation of calculating a loss function according to the embodiments) can be omitted, the back-propagation process and/or the loss function disclosed in this specification ( The operation of calculating the loss function, etc., should not be construed as a conventional or routine construct.

9 to 12 show comparison of performance of an open-domain conversation model of an electronic device with other open-domain conversation models according to embodiments.

9 is a diagram illustrating effects of an electronic device according to embodiments.

Referring to FIG. 9 , 'Bi-encoder 256M' and 'Blender 90M' may be examples of a baseline retrieval model and a baseline generative model. Referring to FIG. 8 , 'RetNRef', 'RetNRef_α', and 'MatToGen' represent examples of generation model units according to embodiments, and CORGE represents search model units according to embodiments. Specifically, RetNRef is an example of a generation model unit to generate an answer, and can connect context information given as an input and a search model unit. RetNRef_α is a conversational search version of RetNRef, which can contain constructs that adopt mixed constructs to avoid simply ignoring retrieved examples (α = 0:5). MatToGen can be a model that extracts meaningful tokens from the example answer(s) and feeds them to the generator. In FIG. 8, RAG and KIF represent an open-domain conversation model based on a Knowledge-grounded Generative model. FIG. 8 shows an experiment conducted by combining 'RetNRef', 'RetNRef_α', and 'MatToGen' with an electronic device (or a search model unit of the electronic device) according to embodiments and comparing performance for each of the two models.

'Appropriateness' in FIG. 9 may mean an index that measures how fluent and logical an answer is and how well it matches the context (context), and 'Informativeness' is how well a generated answer is based on context information. It may refer to indicators that measure whether meaningful information is included, and each indicator is described in Wetson et al. (2018).

Figure 9 summarizes the comparison results of the two models. When RetNRef and MatToGen adopt the search model part (CORGE) according to the embodiments, the effect of surpassing all baselines except for the case of RetNRef + CORGE vs. KIF for informativeness is provided. Specifically, RetNRef + CORGE and MatToGen + CORGE combining search model units according to the embodiments provide better performance than RetNRefα and MatToGen in both metrics, respectively. In particular, it can be seen that MatToGen + CORGE outperforms Biencoder 256M and greatly outperforms Blender 90M, while MatToGen is inferior to Bi-encoder 256M and Blender 90M. In addition, it can be confirmed that CORGE, a search model unit according to embodiments, has a high win rate of RetNRefa against Blender 90M. These evaluation results show that CORGE, a search model unit according to the embodiments, leads the existing model-based generative model to create a more fluent and informative generative model.

10 is a diagram illustrating effects of an electronic device according to embodiments.

10 is automatic evaluation metrics PPL (Perplexity), Dist-n, BLEU (Papineni et al., 202 ) is shown.

는 골드 응답 정보가 예시 답변으로 주어졌을 때의 상황을 가정한 조건부 확률

를 이용한다. 두 번째로, 2)

는, z가

을 이용하여 검색된 예시 답변을 나타내는 조건부 확률

를 이용한다.

가 작다는 것은, 골드 응답 정보가 예시 답변으로 제공되었을 때 예시-기반 생성 모델이 골드 응답 정보를 잘 예측한다는 것을 의미한다.

이 작다는 것은 예시-기반 생성 모델이 골드 응답 정보를 예측하기 위하여 제공된 예시 답변을 잘 활용한다는 것을 의미한다.PPL is an index that measures how well a model predicts an answer based on the presented input context. A low PPL indicates that the model predicts an answer well. To analyze how much the example-based generative model utilizes retrieved example answers, we can use two modified indicators of PPL utilizing conditional probabilities when example answers are provided. First, 1)

is a conditional probability assuming the situation when gold response information is given as an example answer

Use Second, 2)

is, z is

Conditional probability representing the example answer retrieved using

Use

When is small, it means that the example-based generative model predicts the gold response information well when the gold response information is provided as an example answer.

This smallness means that the example-based generative model makes good use of the example answers provided to predict gold response information.

Dist-n follows Li et al., 2016, and represents the ratio of distinct n-grams to the total number of n-grams for all generated answers, It may be an indicator of the diversity of generated answers.

BLEU may be an indicator for measuring the degree of token overlap between the provided example answer and the generated answer pair (z, r). A high BLEU score indicates that the generative modeling unit copied or referenced many parts from the example answers provided in generating the answers.

According to FIG. 10 , a configuration in which RetNRef and a search model unit of an electronic device according to embodiments are combined, and a configuration in which MatToGen and a search model unit of an electronic device according to embodiments are combined show PPL_retrieve lower than Blender 90M. This means that the CORGE-trained example-based generative model of the electronic device according to the embodiments predicts the Gold response better than Blender 90M using provided examples. The configuration combining RetNRef and the search model unit of the electronic device according to the embodiments has smaller PPL_gold and PPL_retrieve degrees than RetNRef, which means that the configuration combining RetNRef and the search model unit of the electronic device according to the embodiments is RetNRef the provided example answer. It can be inferred that it is better used.

RetNRef has a lower PPL_gold than a configuration in which RetNRef and a search model unit of an electronic device according to embodiments are combined, but RetNRef has a PPL_retrieve higher than a configuration in which RetNRef and a search model unit of an electronic device according to embodiments are combined. This result shows that RetNRef does not make good use of retrieved examples, except when provided as example answers retrieved with gold response information. According to this observation, it can be inferred that RetNRef generates highly overfitted responses to selected examples that arise from utilizing gold response information as examples in the training phase. In addition, the model of the electronic device according to the embodiments alleviates the overfitting problem, which is also shown in combination with MatToGen. Compared to Blender 90M, the higher Dist-n of the configuration combining RetNRef and the search model unit of the electronic device according to the embodiments, and the configuration combining MatToGen and the search model unit of the electronic device according to the embodiments, is the electronic device according to the embodiments. We show that the device's model generates more diverse responses than the vanilla generative model. In addition, since the configuration combining RetNRef and the search model unit of the electronic device according to the embodiments has a higher Dist-n than RetNRef, the generator can help diversify the response by using example answers. Simply, RetNRef is the only model that achieves a Dist-n comparable to Bi-encoder 256M, a vanilla search model, but considering the gap between PPL_gold and PPL_retrieve, it can be confirmed that it is overfitted to examples, resulting in poor relevance and informativeness.

The average BLEU score implicitly measures the overlap between retrieved example answers and generated answers, so a higher BLEU level indicates that the generator is more dependent on the retrieved model. RetNRef shows a negligible BLEU score, reaffirming that the model is making little use of the retrieved model. In addition, RetNRefa and MatToGen each have a higher BLEU score than a configuration combining RetNRef and a search model unit of an electronic device according to embodiments, and a configuration combining MatToGen and a search model unit of an electronic device according to embodiments, which is Confirm that the configuration of the electronic device according to the examples does not depend more on the example answers retrieved by the search model unit.

11 is an example for proving the effect according to the above-described result.

A configuration combining Bi-encoder 256M, Blender90M, RetNRef, RetNRefa, and RetNRef with the search model unit of the electronic device according to the embodiments, and the KIF and RAG models represent answers generated in comparison to the input context.

12 illustrates effects according to the above results of a model of an electronic device according to embodiments in comparison to a knowledge-based generation model.

Specifically, the standard deviation of the normalized searcher scores becomes smaller when jointly training the searcher on the sample-based generative model. Here, 5 samples (k = 5) are used for each training instance. 'Ours' in FIG. 11 represents RetNRef + CORGE and 'joint' represents training of the search model unit together with the generative model unit.

Meanwhile, knowledge-grounded generation is also described.

Knowledge-based generative models that use retrieved results (e.g., related documents identified from Wikipedia) to generate answers are knowledge-intensive NLP tasks (e.g., open-domain question and answer). ) was proposed to perform. Knowledge-based production is similar to example-based production. However, the main difference is that knowledge-based generative models extract knowledge from external resources to generate answers. Guu et al. (2020) demonstrate the efficiency of pre-training the knowledge extraction unit with a large-scale language model for answering open-domain questions. Lewis et al. (2020) prove that knowledge-based generative models can generate more informative and diverse sentences than pure generative models in various aspects of knowledge-intensive NLP tasks. Fan et al. (2021) similarly proposes a knowledge-based generative model for answer generation, but is not focused on open-domain conversations. In this specification, we show that existing knowledge-based generative models cannot be directly applied to open-domain conversations.

를 사용하여 외부 지식을 가져오지만 개방형 도메인 대화의 일대다 특성으로 인해 생성기는 다시 검색된 예시 답변을 기본 예제로 무시할 수 있다. The automatic evaluation result according to FIG. 12 confirms that the knowledge-grounded models are ignoring the example answers. RAG and KIF's PPLgold, PPLretrieve, and Dist-n have similar degrees to Blender 90M, indicating that example answers do not provide useful information while generating responses. Also, the average BLEU score is low indicating that there is little overlap between the examples retrieved and the responses generated. These results stem from the difference between open domain conversations and knowledge base creation tasks. While learning and training a knowledge-based generative model

to bring in external knowledge, but due to the one-to-many nature of open domain conversations, generators may ignore back-retrieved example answers as default examples.

의 표준 편차는 거의 0에 가까워진다. 표준 편차가 작을수록 관련성 점수가 평평해지고 있음을 의미한다. 지식 기반 생성 모델은 검색기와 생성기를 공동으로 훈련하면 지식 집약적 NLP 작업에서 성능이 향상된다는 것을 경험적으로 보여주었지만(Lewis et al., 2020), 공개 도메인 대화에서 검색된 예시 답변은 무시된다. 따라서 검색자는 정보가 없는 관련성 점수를 생성하는 방법을 학습한다. 결과적으로 검색기가 축소되어 검색기가 생성기에 부적절한 전형을 반환할 수 있다(도 10의 KIF 및 RAG의 예에도 표시됨). 실시예들에 따른 전자 장치와 함께 검색 모델부를 훈련하면 도 11와 같이 리트리버 점수가 평평해지며 RAG에서도 경험한 것처럼 리트리버의 사소한 붕괴를 경험적으로 관찰할 수 있다. 따라서 실시예들에 따른 전자 장치는 검색 모델부를 공동으로 훈련시키지 않을 수도 있다. 지식-기반 생성 모델들이 경험적으로 보여준 바와 같이 결합하여 훈련되는 검색부 및 생성부는 지식-집약적인 NLP 작업 내 성능을 향상시키지만 (Lewis et al., 2020), 오픈-도메인 대화에서는 검색된 예시들이 무시된다; 따라서 검색부는 정보적이지 않은 연관성 점수를 생성한다. In addition, training the search model unit (retriever) according to the embodiments together with the generator results in the retriever being trapped in a local minimum. As shown in Figure 4, the normalized relevance score calculated by the searchers in the RAG when they are jointly trained.

The standard deviation of is close to zero. A smaller standard deviation means a flatter relevance score. Knowledge-based generative models have shown empirically that jointly training searchers and generators improves performance on knowledge-intensive NLP tasks (Lewis et al., 2020), but example answers retrieved from public domain conversations are ignored. Thus, searchers learn how to generate uninformative relevance scores. As a result, the searcher may be collapsed, causing the searcher to return inappropriate exemplars to the generator (also shown in the example of KIF and RAG in Figure 10). When the search model unit is trained with the electronic device according to the embodiments, the retriever score is flattened as shown in FIG. 11, and minor collapse of the retriever can be empirically observed as experienced in RAG. Accordingly, the electronic device according to embodiments may not jointly train the search model unit. As knowledge-based generative models have shown empirically, jointly trained retrieval and generator parts improve performance in knowledge-intensive NLP tasks (Lewis et al., 2020), but retrieved examples are ignored in open-domain conversations. ; Accordingly, the search unit generates non-informative relevance scores.

Hereinafter, an ablation study (verification of ablation training) of CORGE (electronic device) disclosed in this specification will be described.

Here, we show that the Relevance Score (RS) and kNE clustering algorithms lead the generator to actively use examples, and the Jaccard Filter (JF) to mitigate the overfitting problem. Referring to FIG. 9B, PPL_retrieve of RetNRef + CORGE is low compared to other restrained comparison groups, which shows that each component contributes to exemplifying the answer. RetNRef + CORGE - RS and RetNRef + CORGE - kNE show high levels of PPL_retrieve and PPL_gold, which shows that RS and kNE help the generator to exploit examples in the process of generating answers. RetNRef + CORGE - JF provides a signal that is strong against overfitting and shows significantly lower values for PPL_gold, but in contrast, higher PPL_retrieve values. Dist-n indicates that the model proposed herein generates the most diverse answers within the models except for RetNRef + CORGE - JF, and RetNRef + CORGE - JF indicates excessive copying of tokens from the retrieved example. The average BLEU score also showed the same trend, reaffirming the effect of the CORGE component.

In conclusion, in this specification, we propose a training model applicable to an example-based generative model that can maximize performance in terms of relevance and informativeness. The training method proposed in this specification selects examples that are semantically similar but are reasonably far from the gold response and weights the examples based on the relevance score in the search unit, thereby generating existing examples-based. The disadvantages of the models can be mitigated. Through extensive analysis including pairwise human evaluation, it is confirmed that the training method proposed in this specification improves the performance of existing example-based generative models.

13 shows an example of a method for calculating a relevance score according to embodiments.

및 후보 레이블의 인코딩된 벡터

는 도 12의 1200에 나타난 바와 같이 나타낼 수 있다. First, encoded vectors of an input context (eg, context information according to embodiments) and a candidate label (eg, an example answer according to embodiments, a selected example answer) (For example, as shown in FIG. 13 , vectors encoded by the operation of the encoders 1200a and 1200b included in the electronic device according to the embodiments) are checked. For example, the encoded vector of the input context

and the encoded vector of candidate labels.

may be represented as shown in 1200 of FIG. 12 .

및

는 도 13에서 설명되는 도 13에 나타난 동작(들)이 기 훈련된 두 트랜스포머(transformer)들을 의미할 수 있으며, 처음에는 동일한 가중치로 시작하지만 미세 조정(fine tuning) 과정에서 업데이트될 수 있다.

는 각 트랜스포머 T의 출력 결과를 의미할 수 있으며,

는 벡터들의 시퀀스(sequence)를 하나의 벡터로 감소시키는 함수를 의미할 수 있다. 입력 컨텍스트 및 후보 레이블이 각각 별도로 인코딩되므로, 세그먼트 토큰(segment token)들은 각각 0이다. 사전 훈련 중에 수행된 것과 유사하게 입력과 레이블 모두 특수 토큰 [S]로 둘러싸여 있으므로 h1은 [S]에 해당될 수 있다. 벡터들의 시퀀스(sequence)를 하나의 벡터로 감소시키는

는 예를 들어, 다음 방법들 중 하나 이상을 포함할 수 있다. 1) 트랜스포머(특수 토큰 [S]에 대응하는)의 첫 번째 출력물을 선택하는 방법, 2) 최초 m < N 개의 출력물 벡터들에 대한 평균 또는 전체 출력물 벡터들에 대한 평균으로 결정하는 방법. 도 13에 따른

및

는 도 13의 1200c 및 1200d에 나타난 구성요소 일부 또는 전부를 포함할 수 있다.here,

and

may refer to two transformers previously trained in the operation(s) shown in FIG. 13 described in FIG. 13, and initially start with the same weight, but may be updated in a fine tuning process.

May mean the output result of each transformer T,

may mean a function that reduces a sequence of vectors into one vector. Since the input context and candidate label are encoded separately, the segment tokens are each 0. Similar to what was done during pre-training, both inputs and labels are surrounded by special tokens [S], so h1 can correspond to [S]. reducing a sequence of vectors to one vector

may include, for example, one or more of the following methods. 1) A method of selecting the first output of the transformer (corresponding to the special token [S]), 2) A method of determining the average of the first m < N output vectors or the average of all output vectors. according to Figure 13

and

may include some or all of the components shown in 1200c and 1200d of FIG. 13 .

의 점수(score)는 도 12의 1201에 나타난 수학식 및 도 13의 1201a에 나타난 바와 같이 닷-곱셈(dot product)에 의해 계산될 수 있다. 여기서, 네트워크(예를 들어, 도 13에서 설명하는 네트워크)는 로짓(logit)들이 각각

이고,

이 올바른 레이블이고 나머지는 훈련 세트에서 선택될 수 있는 경우로, 크로스 엔트로피(cross-entropy) 손실이 최소화되도록 훈련된 네트워크일 수 있다.candidate label

The score of can be calculated by dot product as shown in Equation 1201 of FIG. 12 and 1201a of FIG. 13 . Here, the network (eg, the network described in FIG. 13) has logits, respectively

ego,

is the correct label and the rest can be selected from the training set, which can be a network trained to minimize cross-entropy loss.

Therefore, the relevance score according to the embodiments is obtained by using the score of the candidate label calculated by using the context information according to the embodiments as an input context and the example answer according to the embodiments as a candidate label. can mean In addition, association scores according to embodiments, for example, Humeau, Samuel, et al. (2019) may also mean a score generated by the encoder (s) (eg, Bi-encoder, Poly-encoder, etc.).

A recording medium according to various embodiments of the present disclosure may include a computer-readable non-transitory recording medium recording a program for executing a method of generating conversation information using an example-based generation model in a computer.

On the other hand, in the present specification and drawings, preferred embodiments of the present disclosure have been disclosed, and although specific terms have been used, they are only used in a general sense to easily describe the technical content of the present disclosure and help understanding of the present invention. It is not intended to limit the scope of the disclosure. In addition to the embodiments disclosed herein, it is obvious to those skilled in the art that other modified examples based on the technical spirit of the present disclosure may be implemented.

An electronic device or terminal according to the above-described embodiments includes a processor, a memory for storing and executing program data, a permanent storage unit such as a disk drive, a communication port for communicating with an external device, a touch panel, and a key ), user interface devices such as buttons, and the like. Methods implemented as software modules or algorithms may be stored on a computer-readable recording medium as computer-readable codes or program instructions executable on the processor. Here, the computer-readable recording medium includes magnetic storage media (e.g., read-only memory (ROM), random-access memory (RAM), floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM) ), and DVD (Digital Versatile Disc). A computer-readable recording medium may be distributed among computer systems connected through a network, and computer-readable codes may be stored and executed in a distributed manner. The medium may be readable by a computer, stored in a memory, and executed by a processor.

This embodiment can be presented as functional block structures and various processing steps. These functional blocks may be implemented with any number of hardware or/and software components that perform specific functions. For example, an embodiment is an integrated circuit configuration such as memory, processing, logic, look-up table, etc., which can execute various functions by control of one or more microprocessors or other control devices. can employ them. Similar to components that can be implemented as software programming or software elements, the present embodiments include data structures, processes, routines, or various algorithms implemented as combinations of other programming constructs, such as C, C++, Java ( It can be implemented in a programming or scripting language such as Java), assembler, or the like. Functional aspects may be implemented in an algorithm running on one or more processors. In addition, this embodiment may employ conventional techniques for electronic environment setting, signal processing, and/or data processing. Terms such as “mechanism”, “element”, “means” and “composition” may be used broadly and are not limited to mechanical and physical components. The term may include a meaning of a series of software routines in association with a processor or the like.

The foregoing embodiments are only examples, and other embodiments may be implemented within the scope of the claims described below.

Claims (10)

A conversation model training method in an electronic device,
checking first context information;
identifying a first response set corresponding to the first context information based on a first model;
identifying a response subset selected from the first response set based on Gold response information corresponding to the first context information; and
training the second model based on the first context information and the response subset; including,
How to train a dialog model. According to claim 1,
The response subset is selected from among candidate responses identified based on the Gold response information and a clustering algorithm.
How to train a dialog model. According to claim 2,
The response subset is selected by excluding at least one answer corresponding to a specific range from a value corresponding to the appropriate response information in an embedding space among the candidate responses.
How to train a dialog model. The method of claim 1, wherein the dialog model training method
Further comprising setting weight information based on the first context information for each response included in the response subset,
The step of learning the second model is learning the second model based on the set weight information,
How to train a dialog model. The method of claim 4, wherein the weight information is set based on a relevance score for each answer in the response subset,
The relevance score for the answer is calculated based on a value corresponding to the first context information and a value corresponding to the answer in the embedding space.
How to train a dialog model. 5. The method of claim 4 , wherein the second model checks second context information for the conversation information obtained from the user, and provides suitable response information for the second context information based on the second context information.
How to train a dialog model. According to claim 1,
The first context information includes at least one piece of conversation information obtained from a user.
How to train a dialog model. The method of claim 5, wherein the second model is learned by performing a backpropagation operation using a loss function calculated based on the weight information,
The weight information is calculated by normalizing the relevance score for each answer,
How to train a dialog model. An electronic device performing a method for generating conversation information,
a memory in which at least one program is stored; and a processor; including,
The processor identifies first context information, identifies a first response set corresponding to the first context information based on a first model, and an appropriate response corresponding to the first context information based on a second model. information, identifying a response subset selected from the first response set based on the appropriate response information, and learning the second model based on the first context information and the response subset.
electronic device. As a non-transitory computer-readable storage medium,
a medium configured to store computer readable instructions;
When the computer readable instructions are executed by a processor, the processor:
checking first context information;
identifying a first response set corresponding to the first context information based on a first model;
checking appropriate response information corresponding to the first context information based on a second model;
identifying a response subset selected from the first response set based on the appropriate response information; and
training the second model based on the first context information and the response subset; to do,
A non-transitory computer-readable storage medium.

Images