KR102384694B1 - Natural language processing method and natural language processing device using neural network model and non-neural network model - Google Patents

Abstract

The present invention provides a natural language processing method, comprising: extracting advanced semantic features by performing natural language recognition of first source data through an LDA model provided in the apparatus; extracting natural language features including the advanced semantic features by using second source data; performing learning of a non-neural network model for natural language recognition using the advanced semantic feature and the natural language feature; performing learning of a neural network model for natural language recognition using the second source data; performing additional learning of the non-neural network model using result data learned by the neural network model; and forming a hybrid model for natural language recognition using both the non-neural network model and the neural network model.

Description

NATURAL LANGUAGE PROCESSING METHOD AND NATURAL LANGUAGE PROCESSING DEVICE USING NEURAL NETWORK MODEL AND NON-NEURAL NETWORK MODEL

The present invention relates to a natural language processing method, and more particularly, to a natural language processing method and a natural language processing apparatus using a neural network model and a non-neural network model.

Natural language processing (NLP) is an element technology that includes natural language analysis, understanding, and generation, and is applied to various fields such as information retrieval, machine translation, and Q&A.

Natural language processing uses techniques such as natural language analysis, natural language understanding, and natural language generation. Natural language analysis can be divided into four types: morphological analysis, syntactic analysis, semantic analysis, and pragmatic analysis according to the degree. Natural language understanding is a technology that makes a computer operate according to input given in natural language, and natural language generation is a technology that converts the contents of videos or tables into natural language that humans can understand.

Recently, a neural network model has been used in such natural language processing.

Although such a neural network model provides improved performance in semantic analysis in natural language processing, there is a problem in that it cannot provide high accuracy when there is little source data and is driven by inconsistent behavior.

Unexamined Patent Publication No. 10-2019-0046631

The problem to be solved by the present invention is to provide a natural language processing method and a natural language processing apparatus that perform natural language processing with high accuracy and consistency even when using a small amount of source data by using a neural network model and a non-neural network model together.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The present invention for solving the above problems in a natural language processing method,

extracting advanced semantic features by performing natural language recognition of first source data through the LDA model provided in the device;

extracting natural language features including the advanced semantic features by using second source data; performing learning of a non-neural network model for natural language recognition using the advanced semantic feature and the natural language feature; performing learning of a neural network model for natural language recognition using the second source data; performing additional learning of the non-neural network model using result data learned by the neural network model; and forming a hybrid model for natural language recognition using both the non-neural network model and the neural network model.

In addition, the step of extracting advanced semantic features may include: acquiring a language output value including unique identification information corresponding to at least one topic included in the first source data and an occurrence probability of the topic; and determining a meaning recognition rate corresponding to the first source data and a structure recognition rate of the first source data based on the occurrence probability of the language output value with respect to the number of occurrences of the topic.

In addition, the extracted natural language feature may include a discourse-based feature (Disco) including a higher structure of a sentence of a natural language included in the second source data.

In addition, the extracted natural language features may include features of phrases and clauses constituting the natural language included in the second source data and syntax features (Synta) corresponding to the structures of the phrases and clauses.

In addition, the extracted natural language feature may include a variant feature lxSem corresponding to the use of variants of nouns, verbs, adjectives, and adverbs of the natural language included in the second source data.

The natural language processing method according to an embodiment of the present invention comprises the steps of extracting the natural language feature including a transforming feature (lxSem) corresponding to the transforming use of a noun, a verb, an adjective, and an adverb of a natural language included in third source data ; and performing learning of the non-neural network model based on the natural language feature.

In addition, the extracted natural language feature may include a surface feature (ShaTr) corresponding to the syllable feature of the natural language included in the second source data.

In addition, in the hybrid model forming step, the hybrid model may be formed using the non-neural network model that performs learning in parallel with the neural network model using RNN.

In addition, in the hybrid model forming step, a first weight and a second weight corresponding to each of the non-neural network model and the neural network model are determined based on the amount of text included in the first source data and the second source data, and the The hybrid model may be formed by reflecting the result data output by the non-neural network model and the result data output by the neural network model based on the first weight and the second weight.

An apparatus for performing natural language processing according to an embodiment of the present invention, comprising: a memory for storing first source data, second source data, and an LDA model; Including; at least one processor to communicate with the memory;

The at least one processor performs natural language recognition of the first source data through the LDA model to extract advanced semantic features, and uses the second source data to perform natural language features including the advanced semantic features. extracting , learning of a non-neural network model for natural language recognition using the advanced semantic feature and the natural language feature, learning of a neural network model for natural language recognition using the second source data, and Further learning of the non-neural network model is performed using the result data learned by the neural network model,

A hybrid model for natural language recognition may be formed using both the non-neural network model and the neural network model.

Other specific details of the invention are included in the detailed description and drawings.

The natural language processing method and natural language processing apparatus according to an embodiment of the present invention can perform natural language processing with high accuracy and consistency even when using a small amount of source data by using a neural network model and a non-neural network model together.

The natural language processing method and natural language processing apparatus according to an embodiment of the present invention utilizes a statistical method and a latent dirichlet allocation (LDA) topic modeling technique in semantic analysis to derive text recognition difficulty and thus efficiently It can perform natural language processing.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a diagram illustrating a block diagram of a natural language processing apparatus according to an embodiment of the present invention.
2 and 3 are diagrams for explaining the operation of a neural network model and a non-neural network model according to an embodiment of the present invention.
4 is a diagram for explaining a neural network model according to an embodiment of the present invention.
5A and 5B are diagrams for explaining advanced semantic analysis according to an embodiment of the present invention.
6 to 10 are flowcharts according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

In the present specification, source data may refer to data including text data used in natural language processing.

In the present specification, the advanced semantic feature may mean a feature extracted through semantic analysis of source data.

In the present specification, the hybrid model may refer to a natural language processing model using both a neural network model and a non-neural network model.

In the present specification, the language output value may refer to output data in which identification information corresponding to a subject extracted from source data and an occurrence probability are matched.

In the present specification, the meaning recognition rate may refer to a rate at which the natural language processing apparatus recognizes the meaning of text included in source data.

In the present specification, the structure recognition rate may refer to the structure of a paragraph formed by text on the source data.

In the present specification, the transformation characteristic may mean a characteristic of the form of a language in which text used for source data is transformed from a commonly used language.

In the present specification, sematic richness may mean an index corresponding to the amount of text having a meaning on text included in source data.

In the present specification, semantic clarity may refer to a degree to which each subject of text included in source data is distinguished.

In the present specification, noise of source data may mean a ratio corresponding to text corresponding to unnecessary meaning in text included in source data.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a control block diagram of a natural language processing apparatus 10 according to an embodiment of the present invention.

Referring to FIG. 1 , a natural language processing apparatus according to an embodiment of the present invention may include a memory, at least one processor, and a communication unit.

Meanwhile, the natural language processing device may receive several source data D11, D12, and D13.

The first source data D11 may mean a database including advanced meaning.

Also, the second source data D12 may mean a database including normal text.

Also, the third source data D13 may refer to a database in a form transformed from a common language.

The natural language processing apparatus may perform both the learning of the non-neural network model and the learning of the neural network model by using the data included in the third source data D13.

The communication unit 130 may receive source data from the database.

The communication unit 130 may include one or more components that enable communication with an external device, and may include, for example, at least one of a short-range communication module, a wired communication module, and a wireless communication module.

The memory 110 may store various data for natural language processing including the source data D11, D12, and D13.

The memory 110 is a non-volatile memory device or RAM (such as a cache, read only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) and flash memory (Flash memory). It may be implemented as at least one of a volatile memory device such as a random access memory, a hard disk drive (HDD), or a storage medium such as a CD-ROM, but is not limited thereto.

The at least one processor 120 may include a natural language recognition unit 121 , a neural network model learning unit 122 , and a non-neural network model learning unit 123 .

In addition, the at least one processor 120 may include a central processor 124 for performing the above-described module operations.

In addition, the at least one processor 120 may include a central processor 124 capable of performing the operations of the natural language recognition unit 121, the neural network model learning unit 122, and the non-neural network model learning unit 123 described above. there is.

The at least one processor 120 may extract advanced semantic features by performing natural language recognition of the first source data D11 through the preformed LDA model of the first source data D11.

The LDA model may mean latent Dirichlet allocation (LDA).

The advanced semantic feature may mean a semantic feature of the text corresponding to the first source data D11.

LDA may refer to one of the probabilistic topic model techniques for describing which topics exist in each document for a given document. A detailed description of the LDA is provided below.

The at least one processor 120 may extract natural language features including advanced semantic features by using the second source data D12 .

That is, the second source data D12 is data used to form learning data required for normal natural language processing, and includes not only advanced semantic features processed as the first source data D11, but also discourse-based features, syntax features, transformation features, and Natural language features including surface features can be extracted. A detailed description related thereto will be provided later.

In addition, the at least one processor 120 may perform non-neural network model learning for natural language recognition using advanced semantic features and natural language features.

On the other hand, at least one processor 120 may perform neural network model learning for natural language recognition using the second source data D12.

However, the at least one processor 120 additionally extracts the natural language feature including the transform feature lxSem by using the database of the third source data D13 including a large number of variant texts, and using this, learning can be performed.

The at least one processor 120 may perform additional learning of the non-neural network model using result data learned by the neural network model.

The at least one processor 120 may then form a hybrid model for natural language recognition using both the non-neural network model and the neural network model. A detailed description related thereto will be described in FIG. 2 .

Meanwhile, the at least one processor 120 may obtain a language output value including unique identification information and an occurrence probability corresponding to at least one subject included in the source data from the LDA model formed in advance by using the source data.

The unique identification number may mean identification information corresponding to a unit having a meaning among texts included in the source data.

Also, the occurrence probability may mean the occurrence probability of the text in units having respective meanings.

The at least one processor 120 may count the number of occurrences of the topic corresponding to each of the language output values.

In addition, the at least one processor 120 may extract only the occurrence probability from each of the language output values and then calculate the occurrence probability of the language output value with respect to the number of occurrences of the topic.

The at least one processor 120 may determine a semantic recognition rate and a structure recognition rate of source data corresponding to the at least one subject based on a language output value for the number of occurrences of the subject.

A detailed operation of the natural language processing device 10 calculating each will be described below.

At least one component may be added or deleted according to the performance of the components of the natural language processing apparatus 10 shown in FIG. 1 . In addition, it will be readily understood by those of ordinary skill in the art that the mutual positions of the components may be changed corresponding to the performance or structure of the system.

Meanwhile, each component illustrated in FIG. 1 refers to software and/or hardware components such as Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC).

2 and 3 are diagrams for explaining the operation of a neural network model and a non-neural network model according to an embodiment of the present invention.

Referring to FIG. 2 , the natural language processing apparatus may perform neural network learning and non-neural network learning using the first source data D21 , the second source data D23 , and the third source data D22 .

Specifically, the natural language processing apparatus may extract the advanced semantic feature F21 through the LDA technique M21 using the first source data D21. The natural language processing apparatus may extract richness, clarity, noise, and total number of discovered topics of each text from a database such as Wikipedia.

Also, the natural language processing apparatus may extract the natural language feature including the discourse-based features Disco and F22 including the upper structure of the sentence of the natural language included in the second source data D23.

The discourse-based feature F22 may include a higher-level dependency structure.

The discourse-based features F22 may include trending features of the text structure at micro and macro levels. The discourse-based feature F22 may include an entity density function (EnDF) and an entity grid (EnGF) function.

The entity density function is related to the degree of recognition according to the number of entities, and the entity grid function is related to the consistency of the corresponding syntax.

The at least one processor may extract natural language features including features of phrases and clauses constituting the natural language included in the second source data D23 and syntax features (Synta, F24) corresponding to the structures of the phrases and clauses.

Syntax feature F24 relates to a longer processing time of the text.

This syntax feature (F24) can affect the overall complexity of the text, which is an important indicator of readability.

Syntax feature F24 may implement several variations, including the number of nouns, verbs, and adverb phrases.

Also, the syntax feature F24 may include the structural shape of the parsed tree in the operation on the average parsed tree height.

Also, at least one processor may extract natural language features including transforming features lxSem and F23 corresponding to transforming use of nouns, verbs, adjectives, and adverbs of natural language included in the second source data D23 .

The transformation feature F23 may include attributes related to the meaning of the vocabulary included in the source data, the difficulty of the word, and the unfamiliarity of the word.

Specifically, the transformation feature F23 may include a degree of transformation of a noun, a verb, an adjective, and an adverb indicating the ratio of each word.

Meanwhile, the deformable feature F23 may be extracted using the previously formed third source data D22.

In addition, at least one processor may perform learning of the non-neural network model M24 by using the deformed feature extracted through the third source data D22.

That is, the processor may perform non-neural network learning (M24) using a database other than the database used for general learning and then use it to perform natural language processing.

Also, the processor may extract the natural language feature including the surface features ShaTr and F25 corresponding to the syllable feature of the natural language included in the second source data D23.

The surface area feature F25 may mean a feature corresponding to difficulty.

The surface feature F25 may include an average number of tokens, syllables, and characters of text included in the source data.

Meanwhile, referring to FIGS. 2 and 3 together, at least one processor may perform neural network learning M22 and M23 using the second source data D23, and the first source data D21 and the second source Non-neural network learning (M24) may be performed using the data D23 and the third source data D32.

On the other hand, the processor may form a hybrid model using both of these models.

Referring to FIG. 3 , the processor performs first and second weights W1 and second corresponding to the non-neural network model M31 and the neural network model M32, respectively, based on the amount of text included in the first source data and the second source data. A weight W2 may be determined.

In general, the neural network model M31 exhibits low performance in a small data set, and when the source data is small, consistency according to each corpus classification method may be lacking.

Therefore, when the amount of source data is small, the weight (W1) corresponding to the non-neural network model (M31) is increased to perform training for smooth natural language processing even when the source data is small, and when the amount of source data is large, the neural network The performance of natural language processing may be improved by increasing the weight corresponding to the model M21.

The processor reflects the result data output by the non-neural network model M31 and the result data output by the neural network model M32 based on the first weight W1 and the second weight W2 to form the hybrid model M33 can do.

The hybrid model M33 formed in this way can perform effective and consistent natural language processing even with a small amount of source data.

In summary, the processor may form the non-neural network learning model M31 by performing non-neural network learning using the first source data, the second source data, and the third source data.

In addition, the processor may form the neural network model M32 by performing neural network learning using the second source data.

On the other hand, if there is a lot of source data required for learning, the processor performs learning mainly on the neural network model (M32), and if the amount of source data required for learning is small, the processor can perform learning mainly on the non-neural network model (M31), and this proportion corresponds to each model. This can be done by changing the weights W1 and W2 to be used.

Meanwhile, the operation of the present invention described with reference to FIGS. 2 and 3 is only one embodiment of the present invention, and the embodiment in which the processor forms a hybrid model to improve natural language processing capability is not limited thereto.

4 is a diagram for explaining a neural network model using an RNN according to an embodiment of the present invention.

In general, a recurrent neural network (RNN) may be used for natural language processing.

A recurrent neural network (RNN) is a type of artificial neural network, and has a characteristic that connections between units have a cyclic structure.

Since this structure allows the state to be stored inside the neural network to model time-varying dynamic features, it is possible to process the input in the form of a sequence using the internal memory, unlike the forward neural network.

Therefore, the recurrent artificial neural network can be applied to processing data with time-varying characteristics, such as natural language recognition. Because of these characteristics, RNNs can be used by processors to perform natural language processing.

That is, the processor may use the neural network model using the RNN in performing learning through the neural network model.

Specifically, the RNN used by the processor can learn sequence information of texts included in source data by bringing continuous sequence information without simply connecting vectors of tokens T41, T42, T43, and T4n.

However, since the RNN concept uses only the value of the previous cell as an input, information may be gradually lost no matter how the value of the previous cell includes the calculation result of the previous cell.

Therefore, when the processor performs learning using the RNN, it has high accuracy for short sentence structures, but does not have accuracy for long sentence structures.

To overcome this, it is necessary to learn a large amount of source data, and when a small amount of source data is learned, smooth natural language processing is difficult.

Therefore, in order to overcome these shortcomings, a hybrid model can be formed using a neural network model using RNN and a non-neural network model that performs learning in parallel.

As described above, the processor may perform non-neural network learning in parallel with a model that performs neural network learning of RNN and fuse them to form a final hybrid model.

Meanwhile, the operation of the present invention described with reference to FIG. 4 is merely an embodiment of an RNN used for natural language processing, and a Long Short Term Memory (LSTM) and a Gated Recurrent Unit (GRU) may be used in addition to the RNN for natural language processing.

5A and 5B are diagrams for explaining advanced semantic analysis according to an embodiment of the present invention.

Referring to FIG. 5A , the processor may extract at least one subject included in the source data from the preformed LDA model using the source data.

LDA (Latent Dirichlet Allocation) may refer to a probabilistic generation model for discrete data.

LDA can be used for text-based materials.

LDA may refer to topic modeling performed through latent semantic indexing (LSI) and probabilistic latent semantic analysis (pLSA).

LDA has several assumptions, the most important of which is the exchangeability of words.

Exchangeability can mean the assumption that the order of words does not matter and only the presence or absence of words is important.

Therefore, in LDA, if the order of words is ignored, the literature can be expressed only with the frequency of words included in it.

Based on this assumption, a mixed model including the interchangeability of words and documents can be presented through LDA. However, it is also possible to extend the assumption of exchangeability of LDA in a way (n-gram) that considers a group of specific words as a unit rather than simply thinking of a single word as a unit.

That is, the processor of the natural language processing apparatus may obtain a language output value by extracting at least one subject independently of an order on the source data.

In the natural language processing apparatus, a unique identification number and occurrence probability may be matched to each subject, and a language output value may be formed based on this.

In Fig. 5a, the occurrence probability of identification number 3 and 0.45 was matched to the first subject, and the probability of occurrence of identification number 7 and 0.25 was matched to the second subject.

In addition, identification number 42 was matched to the third subject, an occurrence probability of 0.2 was matched, and identification number 45 and an occurrence probability 0.1 were matched to the last subject (O51).

In addition, the processor may also derive the occurrence probability of such a topic, but may count the number of occurrences of a topic corresponding to how many topics appear in the actual source data.

The processor may output a language output value in which an identification number and an occurrence probability are matched to each subject.

Subsequently, the processor may extract only the occurrence probability of each topic from the language output value (S51).

The processor may derive the relationship between the number of occurrences of the topic and the probability of occurrence of the topic by calculating the occurrence probability of the language output value with respect to the number of occurrences of the topic (R51).

Meanwhile, FIG. 5B is a graph showing the relationship between the number of occurrences of a topic and the probability of occurrence of a topic.

5B , the processor may determine a semantic recognition rate corresponding to at least one subject and a structure recognition rate of source data based on a language output value for the number of occurrences of the subject, which will be described later.

As described above, the semantic recognition rate may mean the degree to which the natural language processing apparatus recognizes the text itself, and the structure recognition rate may mean the degree to which the natural language processing apparatus recognizes the syntax of the text itself.

First, the processor may calculate the semantic amount R62 of the source data by integrating the probability of occurrence of the language output value with respect to the number of occurrences of the topic with respect to the number of occurrences of the topic.

The semantic amount R62 of the source data may be calculated based on Equation 1 below.

Referring to Equation 1, S _R may mean a semantic amount R52 of the source data.

The semantic amount may mean the amount of text having a meaning on text included in the source data.

p _i may mean the probability of occurrence of a topic corresponding to each topic, and i means the number of occurrences of the topic, which may correspond to the x-coordinate of the graph in FIG. 5B .

Specifically, in the graph presented in FIG. 5B , the area under the base may represent the semantic amount R62 of the source data.

This large amount of meaning (R52) means that the number of texts to be recognized is large.

As the semantic amount of the source data increases, the semantic recognition rate and the structure recognition rate may be determined to decrease.

That is, if the amount of meaning is large, the meaning recognition rate and structure recognition rate of the corresponding source data may be determined to be low, and the difficulty level of the corresponding source data may be determined to be low and readability.

Also, the processor may calculate the semantic clarity C52 of the source data based on the amount of change in the occurrence probability of the language output value with respect to the number of occurrences of the subject.

The semantic clarity C52 of the source data may be determined based on the following equation.

Referring to Equation 2, S _c denotes semantic clarity, max(p) denotes the maximum value of the occurrence probability of each topic, and pi denotes a topic occurrence probability corresponding to the number of occurrences of each topic.

That is, the processor may determine the semantic clarity C52 based on the amount of change in the occurrence probability with respect to the number of occurrences of each topic with respect to the maximum value of the occurrence probability.

When the meaning clarity C52 is large, it means that the meaning of each text is clearly distinguished, and thus it may mean that the meaning recognition rate or structure recognition rate is large.

That is, the processor may determine that there is a positive correlation in which the semantic recognition rate and the structure recognition rate increase as the semantic clarity increases.

Meanwhile, the processor may further consider the noise N52 in determining the subject recognition rate and the structure recognition rate of the source data.

Specifically, the processor may calculate the average value of the number of occurrences of the topic, and calculate the noise N52 of the source data based on the amount of change in the probability of occurrence of the language output value with respect to the number of occurrences of the topic corresponding to or less than the average value of the number of occurrences of the topic. .

The processor may determine the noise N52 of the source data based on Equation 3 below.

는 각 주제의 발생 확률의 평균을 의미할 수 있다.Referring to Equation 3, S _N means the noise (N52) of the source data,

may mean the average of the occurrence probabilities of each subject.

Since the large noise may mean that it is difficult to recognize a meaning in the source data, the processor may determine that the meaning recognition rate and the structure recognition rate decrease as the noise N52 of the source data increases.

The processor calculates the semantic amount (C52), clarity (C52), and noise (N52) of the source data through the above-described operation, and based on this, calculates the semantic recognition rate of the source data or the structure recognition rate of the syntax constituting the source data. can

Meanwhile, calculating the above-described semantic quantity (C52), clarity (C52), and noise (N52) with the language output value output using the LDA model is only an embodiment of the present invention, and based on this, the semantic recognition rate and structure There is no limit to the operation for determining the recognition rate.

6 to 10 are flowcharts according to an embodiment of the present invention.

Referring to FIG. 6 , the natural language processing apparatus may acquire source data ( S601 ).

Thereafter, the natural language processing apparatus may extract advanced semantic features through the LDA (S602).

In addition, the natural language processing apparatus may extract various natural language features in addition to advanced semantic features ( S603 ).

Meanwhile, the natural language processing apparatus may learn the non-neural network model based on the derived data (S604).

Meanwhile, the natural language processing apparatus having obtained the source data may perform learning of the neural network model ( S605 ).

Thereafter, the natural language processing apparatus may form a hybrid model using the non-trained non-neural network model and the neural network model (S606).

Then, referring to FIG. 7 , the natural language processing apparatus may acquire source data ( S701 ).

The natural language processing apparatus may extract advanced semantic features from the source data (S702).

A discourse-based feature may be extracted (S703). Also, the natural language processing apparatus may extract syntax features (S704).

Also, it is possible to extract deformation features from the source data and extract surface features (S705 and S706). In this way, the natural language processing apparatus may perform learning of the non-neural network model based on the characteristics of the extracted data (S707).

Next, FIG. 8 is a flowchart for explaining the features of the above-described neural network model and non-neural network model.

Referring to FIG. 8 , the natural language processing apparatus may acquire source data ( S801 ). Such source data may be learned through a neural network model, and learning may be performed through a non-neural network model. On the other hand, when learning is performed through the neural network model (S802), a lot of source data is required, the consistency of recognition for each corpus is small, and it is often difficult to explain the recognition.

Accordingly, the natural language processing apparatus may perform the non-neural network model learning and the neural network model learning in parallel (S803).

Through this operation, a hybrid model can be formed by harmonizing the neural network model and the non-neural network model (S804).

9 is a flowchart illustrating an operation of determining a feature of advanced semantics in a non-neural network model.

Referring to FIG. 9 , the natural language processing apparatus may acquire source data ( S901 ).

Thereafter, the natural language processing apparatus may obtain a language output value including a subject, identification information, and an occurrence probability ( S902 ).

Meanwhile, the natural language processing apparatus may count the number of occurrences of the topic (S903).

In addition, it is possible to extract the occurrence probability corresponding to the number of occurrences of each topic (S904). Thereafter, it is possible to determine the amount of meaning (Richness), clarity (Clarity), and noise (Noise) corresponding to each topic (S905).

Finally, the natural language processing apparatus may determine a semantic recognition rate and a structure recognition rate of the corresponding source data (S906).

Referring to FIG. 10 , the natural language processing apparatus may calculate a semantic amount S1002 , a clarity S1003 , and a noise S1004 to determine advanced semantic features using source data S1001 , respectively. The natural language processing apparatus may determine a subject recognition rate and a structure recognition rate of the source data (S1005). The natural language processing apparatus may perform non-neural network learning (S1006).

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

10: natural language processing unit
110: memory
120 : processor
130: communication department

Claims (10)

A method for natural language processing performed by a device, comprising:
Extracting advanced semantic features including semantic quantity, semantic clarity, and noise corresponding to the first source data by performing natural language recognition of the first source data through the LDA model provided in the device ;
extracting natural language features including the advanced semantic features by using second source data;
performing learning of a non-neural network model including at least one of an SVM for natural language recognition, a Naive Bayes Classifier (NBC), and a random forest using the advanced semantic feature and the natural language feature;
performing learning of a neural network model for natural language recognition using the second source data;
performing additional learning of the non-neural network model using result data learned by the neural network model; and
and forming a hybrid model for natural language recognition using both the non-neural network model and the neural network model.
delete According to claim 1,
The extracted natural language feature includes a discourse-based feature (Disco) including a higher-order structure of a sentence of a natural language included in the second source data.
According to claim 1,
The extracted natural language features include features of phrases and clauses constituting the natural language included in the second source data, and syntax features (Synta) corresponding to the structures of the phrases and clauses.
According to claim 1,
The extracted natural language feature includes a variant feature (lxSem) corresponding to the use of variants of nouns, verbs, adjectives, and adverbs of natural language included in the second source data.
6. The method of claim 5,
extracting the natural language feature including a transforming feature (lxSem) corresponding to the use of transforming nouns, verbs, adjectives, and adverbs of natural language included in third source data; and
The natural language processing method further comprising; performing learning of the non-neural network model based on the natural language feature.
According to claim 1,
The extracted natural language features correspond to the syllable characteristics of the natural language included in the second source data, and include a surface area feature (ShaTr) including tokens, syllables, and average number of texts of the second source data. Way.
According to claim 1,
The hybrid model forming step is
A natural language processing method for forming the hybrid model using the non-neural network model that performs learning in parallel with the neural network model using an RNN.
delete In the apparatus for performing natural language processing,
a memory for storing the first source data, the second source data, and the LDA model;
Including; at least one processor to communicate with the memory;
The at least one processor,
performing natural language recognition of the first source data through the LDA model to extract advanced semantic features including semantic quantity, semantic clarity, and noise corresponding to the first source data (Advanced semantic features);
extracting natural language features including the advanced semantic features using the second source data,
Using the advanced semantic feature and the natural language feature to perform learning of a non-neural network model including at least one of SVM, NBC (Naive Bayes Classifier), and random forest for natural language recognition,
performing learning of the neural network model for natural language recognition using the second source data, and performing additional learning of the non-neural network model using the result data learned by the neural network model;
A natural language processing apparatus for forming a hybrid model for natural language recognition by using both the non-neural network model and the neural network model.

Images