CN110442880B - Translation method, device and storage medium for machine translation - Google Patents

Abstract

The invention discloses a translation method, device and storage medium for machine translation translation, comprising: receiving a source sentence to be translated; performing word segmentation processing on the source sentence; model, the part of speech is integrated into the word vector corresponding to the word, and the fused word vector sequence is obtained; the word vector sequence is input into the encoder-decoder model, and the encoding and decoding results are obtained; for the encoding and decoding results, Result evaluation is performed based on a beam search evaluation function, wherein the beam search evaluation function includes a penalty item based on length comparison and a penalty item based on repeated detection; and a translation is obtained according to the evaluation result. The application of the embodiments of the present invention improves the problems of repeated fragments and omission of source sentences in the translation, and has a wide range of applications, strong pertinence, and high translation quality.

Description

A translation method, device and storage medium for machine translation translation

technical field

The present invention relates to the technical field of machine translation translation improvement, in particular to a translation method, device and storage medium for machine translation translation.

Background technique

Language is the most important carrier of information exchange for human beings. It has a very important impact on the development of the whole society. The method of automatic machine translation has become an urgent need at present. The realization of automatic translation of different languages is controlled by a huge application.

At present, rule-based machine translation methods require professional linguists to formulate a large number of rules, with high labor costs and poor scalability. The machine translation method based on intermediate language needs to formulate a set of common intermediate language, which is too difficult and has low robustness. Although statistical-based machine translation methods have lower labor costs and improved scalability, the translation quality is still poor. The machine translation method based on neural network is the most advanced machine translation method at present, but there is still room for improvement in the quality of the translated text.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a translation method, device and storage medium for machine translation translation, aiming at solving the problem of poor quality of translation generated by the existing machine translation model.

In order to achieve the above object, the present invention provides a translation method for machine translation translation, the method comprising:

Receive the source sentence to be translated;

performing word segmentation processing on the source sentence;

Obtain the part of speech of each word in the participle;

According to the word vector model, the part of speech is integrated into the word vector corresponding to the word, and the fused word vector sequence is obtained;

Inputting the word vector sequence into an encoder-decoder model to obtain an encoding and decoding result;

For the encoding and decoding results, result evaluation is performed based on a beam search evaluation function, wherein the beam search evaluation function includes a penalty item based on length comparison and a penalty item for repeated detection;

A translation is obtained based on the evaluation results.

Further, the specific expression of the beam search evaluation function is:

s(Y, X)=log(P(Y|X))+d(x)+l(x)

where x(Y, X) is the beam search evaluation function, log(P(Y/X)) is the probability function of Y appearing when X appears, d(x) is the penalty term based on repeated detection, and l(x) is the penalty based on length comparison, P is the distribution function;

A penalty term based on the length ratio is added to the beam search evaluation function to solve the problem of partial omissions in the translation;

A penalty term based on duplicate detection is added to the beam search evaluation function to solve the problem of duplicate content in translation.

Further, the specific formula of the repeated detection penalty term d(x) is expressed as:

Among them, c is the index of the current translation word, δ is the range of duplicate detection, ε is the penalty coefficient, y is the matrix corresponding to the candidate translation, y _cj , y _cij are the two matrices for duplicate detection, i, j, to iterate over variables.

Further, the step of evaluating the result based on the beam search evaluation function for the encoding and decoding results includes:

the ratio of the length of the source sentence to the length of the target translation;

Fitting the length ratio through linear regression to obtain a cumulative distribution function;

When both end-of-sentence markers and common words appear in the candidate words of the beam search, the probability that the translation has ended F _X (x) and the probability that the translation has not ended 1-F _X (x) are respectively added to the evaluation function l(x )=θF _X (x), not_EOS, l(x)=θ(1-F _X (x)), EOS, where EOS is the end-of-sentence marker, and θ is a parameter;

When the candidate word is the marker at the end of the sentence, multiply the untranslated probability by the penalty factor as the penalty item;

When the candidate word is not the marker at the end of the sentence, the probability of completing the translation is multiplied by the penalty factor as the penalty item;

The obtained penalty term based on the length ratio is added to the evaluation function of beam search;

The results are evaluated based on the beam search evaluation function.

Further, in the encoder-decoder model, both the encoder part and the decoder part use a bidirectional recurrent neural network.

Further, the step of inputting the word vector sequence into an encoder-decoder model to obtain an encoding and decoding result includes:

inputting the sequence of word vectors into an encoder-decoder model;

The deep learning framework based on the codec converts the sequence of word vectors into sentence vectors;

Based on the decoder, the sentence vector is converted into a sequence of word vectors.

In addition, the present invention also discloses a machine translation translation device, the device includes a processor and a memory connected to the processor through a communication bus; wherein,

The memory is used to store a translation program for machine translation translation;

The processor is configured to execute the translation program of the machine translation translation, so as to realize the translation step of any one of the machine translation translations.

And, a computer storage medium storing one or more programs executable by one or more processors to cause the one or more processors to perform any The translation steps of the machine translation translation described in one item.

By applying the translation method, device and storage medium for machine translation translation provided by the embodiments of the present invention, while effectively constructing vectors to establish semantic associations between different words, the meanings of them under different parts of speech are included and corrected. The beam search evaluation function improves the problem of repeated fragments and missing source sentences in the translation, and has a wide range of applications, strong pertinence, and high translation quality.

Description of drawings

FIG. 1 is a schematic flowchart of an embodiment of the present invention.

FIG. 2 is a schematic structural diagram of an embodiment of the present invention.

FIG. 3 is another schematic structural diagram of an embodiment of the present invention.

FIG. 4 is a schematic diagram illustrating a penalty item algorithm for repeated detection according to an embodiment of the present invention.

FIG. 5 is a schematic diagram illustrating a penalty term algorithm for a length ratio according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of an English-Chinese translation effect of an embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

The language model is a simple, unified and abstract formal system. After the objective facts of the language are described by the language model, they can be automatically processed by the computer. Therefore, the language model is of great significance to the information processing of natural language. It plays an important role in the research of part-of-speech tagging, syntactic analysis, and speech recognition.

In the machine translation problem, both the input original language sentence and the output target language translation can be regarded as sequences, so machine translation can be regarded as a sequence-to-sequence problem. The current mainstream method to solve sequence-to-sequence is the encoder-decoder model. The encoder encodes the original language sentence into a sentence vector, and the decoder re-decodes the sentence vector to obtain the target language translation.

It should be noted that the recurrent neural network RNN is usually used as the encoder and decoder. Recurrent Neural Network (RNN, Recurrent Neural Network), is a classic neural network structure, which contains recurrent neural units, so it can process serializable data and allow the persistence of data information. The RNN will train the current input together with the previous input as parameters and get the output. Bidirectional Recurrent Neural Network (Bi-RNN, Bidirectional Recurrent Neural Network) is an improved network structure based on cyclic neural network. In some tasks, the input of the network is not only related to the past input, but also has a certain relationship with the subsequent input. Therefore, in addition to inputting the forward sequence, the reverse sequence also needs to be input. The bidirectional recurrent neural network is composed of a two-layer recurrent neural network, which supports the simultaneous input of forward sequence and reverse sequence, which effectively improves the performance of the network.

WordVector A general term for a set of language modeling and feature learning techniques in embedded natural language processing (NLP), where words or phrases from a vocabulary are mapped to vectors of real numbers. Conceptually, it involves mathematical embeddings from a one-dimensional space per word to a continuous vector space with lower dimensions. The Skip-gram model is a model structure used to generate a distributed representation of a word when a neural network trains a language model. The Skip-gram model uses the word vector of the current word as input to predict the context of the word. Beam Search Beam search is a heuristic search algorithm that explores a graph by expanding the most promising nodes in a finite set.

Beam search is an optimization of best-first search that reduces its memory requirements. An optimal search is a graph search that ranks all partial solutions (states) according to some heuristic commands that try to predict how close the partial solutions are to the complete solution (goal state). But in beam search, only a predetermined number of the best local solutions are kept as candidates.

See Figures 1-6. It should be noted that the drawings provided in this embodiment are only to illustrate the basic concept of the present invention in a schematic way, so the drawings only show the components related to the present invention rather than the number, shape and the number of components in actual implementation. For dimension drawing, the type, quantity and proportion of each component can be changed at will in actual implementation, and the component layout may also be more complicated.

As shown in Figure 1, the present invention provides a translation method for machine translation translation, and the method includes:

S110: Receive the source sentence to be translated.

S120, performing word segmentation processing on the source sentence;

It can be understood that, word segmentation processing is performed on each sentence in the received source sentence to be translated.

S130, obtaining the part of speech of each word in the word segmentation;

It should be noted that each sentence in the source sentence is segmented, and then each word is tagged with a part of speech tagging tool to obtain the part of speech of each word, and the abbreviation symbol of the corresponding part of speech is obtained by querying the part of speech abbreviation table. Finally, connect the original word and the corresponding part-of-speech abbreviation through the "_" symbol to obtain a word/part-of-speech string and replace the original word in the source sentence.

S140, according to the word vector model, integrate the part of speech into the word vector corresponding to the word, and obtain the fused word vector sequence;

WordVector A general term for a set of language modeling and feature learning techniques in embedded natural language processing (NLP), where words or phrases from a vocabulary are mapped to vectors of real numbers. Conceptually, it involves mathematical embeddings from a one-dimensional space per word to a continuous vector space with lower dimensions.

In an implementation manner of the present invention, all words/part-of-speech strings in the source sentence obtained in steps S120 and S130 are counted and a dictionary is constructed, and then each word/part-of-speech string in the dictionary is indexed and saved. Then convert the word/part-of-speech string in the sentence into an index value, and input the index value sequence represented by each sentence into the skip-gram model for training, and obtain the trained word vector fused with the part-of-speech feature. sequence of word vectors.

Exemplarily, as shown in Figure 2, the input is w(t), and after the skip-gram model is trained, the output is w(t-2), w(t-1), w(t+2), w(t respectively +1).

S150: Input the word vector sequence into an encoder-decoder model to obtain an encoding and decoding result.

It should be noted that the word vector sequence is input into the encoder-decoder model, the trained word vector is used to replace the words of each sentence in the original corpus, and the sentence in the corpus is converted into a word vector sequence. Then, the word vector sequence is sent as input into the encoder-decoder model to obtain the encoding and decoding results. The encoder-decoder model structure is shown in Figure 3.

S160, for the encoding and decoding results, perform result evaluation based on a beam search evaluation function, wherein the beam search evaluation function is included in a penalty item based on length comparison and a penalty item based on repeated detection;

Understandably, beam search is a heuristic search algorithm that explores a graph by expanding the most promising nodes in a finite set. Beam search is an optimization of best-first search that reduces its memory requirements. An optimal search is a graph search that ranks all partial solutions (states) according to some heuristic commands that try to predict how close the partial solutions are to the complete solution (goal state). But in beam search, only a predetermined number of the best local solutions are kept as candidates. In the embodiment of the present invention, the evaluation function of beam search is improved, and a penalty item based on repeated detection and a penalty item based on length ratio are added.

S170, obtaining a translation according to the evaluation result;

Through the encoder-decoder model and beam search, the final translation is obtained.

In an implementation manner of the present invention, the specific expression of the beam search evaluation function is:

s(Y, X)=log(P(Y|X))+d(x)+l(x)

where x(Y, X) is the beam search evaluation function, log(P(Y/X)) is the probability function of Y appearing when X appears, d(x) is the penalty term based on repeated detection, and l(x) is the penalty based on length comparison, P is the distribution function;

A penalty term based on the length ratio is added to the beam search evaluation function to solve the problem of partial omissions in the translation;

A penalty term based on duplicate detection is added to the beam search evaluation function to solve the problem of duplicate content in translation.

It should be noted that, in the embodiment of the present invention, the evaluation function of beam search is improved, and a penalty item based on length ratio and a penalty item based on repeated detection are added. The length ratio penalty item is aimed at the problem that the length of the machine translation translation is too long or too short. The penalty item is obtained by counting the ratio of the length of the source sentence to the length of the translated text, and it is used in the evaluation function of the candidate word by the beam search. The translation is divided into fragments of different sizes for comparison, and the distance between the position where the repeated word appears and the position to be translated are also taken into account. Finally, the obtained penalty term is used in the evaluation function of the candidate word by the beam search. The problem of repeated fragments and omission of source sentences in the translation has been improved, with a wide range of applications, strong pertinence, and high translation quality.

Further, the specific formula of the repeated detection penalty term d(x) is expressed as:

Among them, c is the index of the current translation word, δ is the range of duplicate detection, ε is the penalty coefficient, y is the matrix corresponding to the candidate translation, y _cj , y _cij are the two matrices for duplicate detection, i, j, to iterate over variables.

As shown in Figure 4, the candidate sentences of the entire beam search and the parameters δ and ε in the formula are used as the input of the algorithm. After dividing into multiple segments of different sizes for comparison, the respective penalty terms are calculated respectively, and finally the weighted accumulation is performed. Figure 5 This paper uses the current candidate word, the value of the cumulative distribution function calculated by the current length F _X (x), and the parameter θ in the formula as the input of the algorithm. First, whether the candidate word is EOS is obtained by vector operation, if it is, it is 1, 0 otherwise. Then through the form of dot product, the value of l(x) in the formula is obtained.

In an implementation manner of the present invention, the step of evaluating the encoding and decoding results based on the beam search evaluation function includes:

the ratio of the length of the source sentence to the length of the target translation;

Fitting the length ratio through linear regression to obtain a cumulative distribution function;

When both end-of-sentence markers and common words appear in the candidate words of the beam search, the probability that the translation has ended F _X (x) and the probability that the translation has not ended 1-F _X (x) are respectively added to the evaluation function l(x )=θF _X (x), not_EOS, l(x)=θ(1-F _X (x)), EOS, where EOS is the end-of-sentence marker, and θ is a parameter;

When the candidate word is the marker at the end of the sentence, multiply the untranslated probability by the penalty factor as the penalty item;

When the candidate word is not the marker at the end of the sentence, the probability of completing the translation is multiplied by the penalty factor as the penalty item;

The obtained penalty term based on the length ratio is added to the evaluation function of beam search;

The results are evaluated based on the beam search evaluation function.

It can be understood that firstly, the length of the source sentence and the length of the target translation are calculated separately, and the length ratio of the source sentence and the target translation is calculated, and then the length ratio obtained by linear regression is fitted to obtain its cumulative distribution function F _X (x) =P(X<x), where,

That is, the ratio of the length of the target translation to the length of the source sentence. When both EOS (sentence end marker) and common words appear in the candidate words of the beam search, add the probability FX(x) that the translation has ended and the probability 1-F _X (x) that the translation has not ended to their Evaluation function l(x)=θF _X (x), not_EOS, l(x)=θ(1−F _X (x)), EOS. When the candidate word is the EOS symbol, multiply the untranslated probability by the penalty factor as the penalty item, and when the candidate word is not the EOS symbol, multiply the translation probability by the penalty factor as the penalty term. Finally, the penalty term based on the length ratio is added to the evaluation function of beam search, as shown in Figure 5.

Through the encoder-decoder model and beam search, the final optimal translation is obtained, as shown in Figure 6.

It should be noted that the codec is a deep learning framework combined with a beam search evaluation function to obtain the final optimal translation, which solves the problem of repeated fragments and missing source sentences. high.

Further, in the encoder-decoder model, both the encoder part and the decoder part use a bidirectional recurrent neural network.

It should be noted that the Bidirectional Recurrent Neural Network (Bi-RNN, Bidirectional Recurrent Neural Network) is an improved network structure based on the cyclic neural network. In some tasks, the input of the network is not only related to the past input, but also has a certain relationship with the subsequent input. Therefore, in addition to inputting the forward sequence, the reverse sequence also needs to be input. The bidirectional recurrent neural network is composed of a two-layer recurrent neural network, which supports the simultaneous input of forward sequence and reverse sequence, which effectively improves the performance of the network.

In an implementation manner of the present invention, the step of inputting the word vector sequence into an encoder-decoder model to obtain an encoding and decoding result includes:

inputting the sequence of word vectors into an encoder-decoder model;

The deep learning framework based on the codec converts the sequence of word vectors into sentence vectors;

Based on the decoder, the sentence vector is converted into a sequence of word vectors.

It can be understood that the encoder-decoder is a deep learning framework, the encoder is used to convert word vector sequences into sentence vectors, and the decoder is used to convert sentence vectors into word vector sequences.

The present invention also provides a translation device for machine translation translation, the device includes a processor and a memory connected to the processor through a communication bus; wherein,

The memory is used to store a translation program for machine translation translation;

The processor is configured to execute the translation program of the machine translation translation, so as to realize the translation step of any one of the machine translation translations.

The present invention also provides a computer storage medium storing one or more programs, and the one or more programs can be executed by one or more processors to cause the one or more processors to execute any one of the The translation steps of the machine translation translation described above.

The above-mentioned embodiments merely illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed in the present invention should still be covered by the claims of the present invention.

Claims (8)

1. a translation method of machine translation translation, is characterized in that, described method comprises: Receive the source sentence to be translated; performing word segmentation processing on the source sentence; Obtain the part of speech of each word in the participle; According to the word vector model, the part of speech is integrated into the word vector corresponding to the word, and the fused word vector sequence is obtained; Inputting the word vector sequence into an encoder-decoder model to obtain an encoding and decoding result; For the encoding and decoding results, result evaluation is performed based on a beam search evaluation function, wherein the beam search evaluation function includes a penalty item based on length comparison and a penalty item for repeated detection; A translation is obtained based on the evaluation results. 2. the translation method of a kind of machine translation translation according to claim 1, is characterized in that, the concrete expression of described beam search evaluation function is: s(Y, X)=log(P(Y|X))+d(x)+l(x) where s(Y,X) is the beam evaluation function, x(Y,X) is the beam search evaluation function, log(P(Y/X)) is the probability function that Y appears when X appears, and d(x) is The penalty term based on repeated detection, l(x) is the penalty based on length comparison, and P is the distribution function; A penalty term based on the length ratio is added to the beam search evaluation function to solve the problem of partial omissions in the translation; A penalty term based on duplicate detection is added to the beam search evaluation function to solve the problem of duplicate content in translation. 3. the translation method of a kind of machine translation translation according to claim 2, is characterized in that, the concrete formula expression of described repetition detection penalty item d(x) is:

Among them, c is the index of the current translation word, δ is the range of duplicate detection, ε is the penalty coefficient, y is the matrix corresponding to the candidate translation, y _cj , y _cij are the two matrices for duplicate detection, i, j, to iterate over variables. 4. the translation method of a kind of machine translation translation according to claim 2 or 3, it is characterized in that, described for described codec result, the step of result evaluation based on beam search evaluation function, comprises: the ratio of the length of the source sentence to the length of the target translation; Fitting the length ratio through linear regression to obtain a cumulative distribution function; When both end-of-sentence markers and common words appear in the candidate words of the beam search, the probability that the translation has ended F _X (x) and the probability that the translation has not ended 1-F _X (x) are respectively added to the evaluation function l(x )=θF _X (x), not_EOS, l(x)=θ(1-F _X (x)), EOS, where EOS is the end-of-sentence marker, and θ is a parameter; When the candidate word is the marker at the end of the sentence, multiply the untranslated probability by the penalty factor as the penalty item; When the candidate word is not the marker at the end of the sentence, the probability of completing the translation is multiplied by the penalty factor as the penalty item; The obtained penalty term based on the length ratio is added to the evaluation function of beam search; The results are evaluated based on the beam search evaluation function. 5 . The translation method for machine translation translation according to claim 1 , wherein, in the encoder-decoder model, both the encoder part and the decoder part use a bidirectional recurrent neural network. 6 . 6. The translation method of a machine translation translation according to claim 1, wherein the step of inputting the word vector sequence into an encoder-decoder model to obtain an encoding and decoding result comprises: inputting the sequence of word vectors into an encoder-decoder model; The deep learning framework based on the codec converts the sequence of word vectors into sentence vectors; Based on the decoder, the sentence vector is converted into a sequence of word vectors. 7. A translation device for machine translation translation, wherein the device comprises a processor and a memory connected to the processor through a communication bus; wherein, The memory is used to store a translation program for machine translation translation; The processor is configured to execute the translation program of the machine-translated translation, so as to realize the steps of the translation method of the machine-translated translation according to any one of claims 1 to 6. 8. A computer storage medium, wherein the computer storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to cause the one or more programs to The processor executes the steps of the translation method of the machine translation translation according to any one of claims 1 to 6.

Images