KR20210082727A - Semantic linking of natural language words with columns and tables in databases - Google Patents

Abstract

The present invention relates to a method of connecting natural language words with a column and a table of a database. The method, which is used to match words of a natural language sentence with a column and a table of a database through a computing device including a microcomputer, includes the following steps of: (a) checking whether natural language words are matched with column or table names of the database, and matching a matched column or table with a corresponding word; (b) when there are words which are not matched in the step (a), checking whether the column or table names are partially matched with the words, and, when the names are partially matched with the words, matching the names with the words; and (c) when there are natural language words which are not even partially matched in the step (b), checking whether the words are matched with values of the database and, when the words and the values are matched, matching the words, and, when the words and the values are not matched, excluding the words from the matching work.

Description

{Semantic linking of natural language words with columns and tables in databases}

The present invention relates to a method of associating natural language words with columns and tables of a database, and more particularly, to a method of associating natural language words with columns and tables of a database to facilitate realization of a database interface for natural language.

In general, the problem of translating natural language into SQL queries is an important issue in that a person without specialized knowledge can query a relational database. Recently, many rules and neural network-based methods for solving SQL query translation problems have been proposed.

zcan : ATHENA: An Ontology-Driven System for Natural Language Querying over Relational Data Stores PVLDB 9(12): 1209-1220 (2016)(선행문헌1)"에서는 자연어를 SQL 질의로 변환하는 규칙 기반 기술 ATHENA를 제안하였다. ATHENA는 전문가가 사전 정의한 온톨로지(ontology) 및 텍스트의 단어와 온톨로지의 개체를 매핑하는 사전을 이용해 자연어의 단어들을 온톨로지의 개체들에 매핑한다. 이렇게 매핑된 개체들 및 자연어를 이용해 OQL(Ontology Query Language) 질의를 생성하고, 다시 이를 SQL 질의로 변환하는 방법으로 SQL 질의 번역을 수행한다. "Diptikalyan Saha, Avrilia Floratou, Karthik Sankaranarayanan, Umar Farooq Minhas, Ashish R Mittal, Fatma

zcan : ATHENA: An Ontology-Driven System for Natural Language Querying over Relational Data Stores PVLDB 9(12): 1209-1220 (2016) (Prior Document 1)” proposed ATHENA, a rule-based technology that converts natural language into SQL queries. ATHENA maps words in natural language to objects in ontology using an ontology predefined by experts and a dictionary that maps words in text and objects in the ontology. Using these mapped objects and natural language, OQL (Ontology Query) Language) The SQL query is translated by creating a query and converting it back to an SQL query.

This method has a problem in that the ontology and dictionary need to be created by an expert in advance to operate, and there is a problem in that the ontology and dictionary must be modified when the database schema is changed.

With the development of neural network technology and the publication of SQL query translation data, neural network-based SQL query translation technology is being actively studied. Recent neural network-based studies can be largely divided into slot filling and context-free grammar-based methods.

The slot filling method assumes a simple SQL query with a specific template and fills each slot of the template through a neural network. "Victor Zhong, Caiming Xiong, Richard Socher: Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning CoRR abs/ 170900103 (2017) (Prior Document 2)” and “Tao Yu, Zifan Li, Zilin Zhang, Rui Zhang, Dragomir R Radev: TypeSQL: Knowledge-Based Type-Aware Neural Text-to-SQL Generation NAACL-HLT (2) 2018 : 588-594 (Prior Document 3)”.

Prior Document 2 published WikiSQL, a dataset consisting of three pairs of a simple SQL query with a specific template and a natural language and database.

Prior Document 3 presented a neural network-based method called TypeSQL, considering the SQL query translation in WikiSQL as a slot filling problem for the first time.

However, as in WikiSQL, the SQL query to which the slot filling method can be applied is very simple, so it is difficult to use it realistically. For example, slot filling methods do not assume various keywords (ORDER BY, GROUP BY, etc.) and cannot process nested queries.

In addition, examples of the context-free grammar-based method include "Wonseok Hwang, Jinyeung Yim, Seunghyun Park, Minjoon Seo: A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization CoRR abs/190201069 (2019) (Prior Document 4)" and "Pengcheng" He, Yi Mao, Kaushik Chakrabarti, Weizhu Chen: X-SQL: reinforce schema representation with context CoRR abs/190808113 (2019) (Prior Literature 5)”.

Prior Documents 4 and 5 above proposed SQLova and X-SQL using transfer learning in neural networks pre-trained in many natural languages, and obtained high accuracy of about 90% in WikiSQL.

The context-free grammar-based method generates the final sentence by leftmost derivation through a neural network based on the context-free grammar of a programming language such as SQL query. All SQL queries can be created, overcoming the limitations of the slot filling method.

"Tao Yu, Michihiro Yasunaga, Kai Yang, Rui Zhang, Dongxu Wang, Zifan Li, Dragomir R Radev: SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-Domain Text-to-SQL Task EMNLP 2018: 1653-1663 (Prior Document 6) )" is the first study to translate SQL queries in a context-free grammar-based method.

Also, "Jiaqi Guo, Zecheng Zhan, Yan Gao, Yan Xiao, Jian-Guang Lou, Ting Liu, Dongmei Zhang: Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation ACL (1) 2019: 4524-4535 (Prior Literature 7)” proposed a rule-based method for linking database schema and words in natural language, and also proposed a neural network method to translate SemQL queries from natural language by proposing an intermediate language, SemQL, and to translate SemQL queries into SQL queries. SQL query translation is performed through a rule-based method.

1 is a flow chart of natural language processing for a general relational database.

Referring to FIG. 1, according to the input of a natural language query as shown in (a), it is converted into an SQL query as shown in (b), and the converted content is searched in a relational database as shown in (c) and the result is output in natural language. .

The problem of creating a natural language interface for a database system is an important issue in that it allows non-experts to easily query the database. The most widely used relational database among database systems is made in the SQL programming language.

Thus, by translating a natural language query into a SQL query, a natural language interface to a relational database can be created.

One of the great difficulties of translating a natural language query into a SQL query is figuring out which columns and tables in the database the natural language query refers to. The reason this is difficult is that the column and table names of the database do not appear as they are in natural language queries, but may appear paraphrased or may not appear directly. In addition, it is difficult to accurately find columns and tables to be used in SQL queries because there may be multiple columns and tables with similar names in the database schema.

Existing rule-based SQL query translation methods used a method in which an expert creates a mapping dictionary in advance, but this causes problems in situations where words not considered in the dictionary appear or the database schema changes. In addition, every database has a fundamental problem that requires great effort by experts.

In particular, it is very difficult to make a matching dictionary for all situations or to learn in advance through machine learning because a lot of words from a specific domain that do not normally appear in natural languages can appear in the actual database.

Therefore, a new method of matching natural language words with database columns and tables is required.

The technical problem to be solved by the present invention is a case in which it is difficult to semantically connect a word with a column and a table, such as when several words appearing in a natural language sentence are matched, only a part of a word is matched, or a word is matched to several columns and tables An object of the present invention is to provide a method of linking natural language words that can be matched with columns and tables of a database.

The present invention relates to a method of linking natural language words with columns and tables of a database for solving the above problems, and the present invention relates to a method of linking natural language words with columns and tables of a database, Computing device including a microcomputer A method of matching a word in a natural language sentence with a column and table of a database using b) if there are words that do not match in step a), checking whether the column or table name partially matches the word, and matching if it partially matches, c) also partially matching in step b) If there is a natural language word that does not match, check whether it matches with the values in the database, match if they match, and exclude from matching if they do not match.

In an embodiment of the present invention, the natural language words in step a) may be extracted through headword extraction from a natural language sentence.

In an embodiment of the present invention, the word matching the column or table in step a) may not perform the partial identity determination in step b).

In an embodiment of the present invention, the word matched with the column or table in steps a) and b) may not be compared with the internal database value in step c).

In an embodiment of the present invention, the step c) includes the steps of c-1) determining whether the word matches the internal value of the database, and c-2) if the result of the c-1) match, the corresponding If the word is a word that is not suitable as a predefined database internal value, a process of not matching may be included.

In an embodiment of the present invention, in step c-2), it may be determined whether a word is not suitable as a predefined database internal value through part-of-speech analysis of a word that matches an internal value in the database.

The method of linking a natural language word to a database column and table of the present invention comprehensively determines the name of the column and table and records in the database, so that several words are matched or only part of a word is matched, or a word is entered in several columns and tables Even when matched, there is an effect that can be effectively connected.

1 is a flow chart of natural language processing for a general relational database.
2 is a flowchart of a method of associating natural language words with columns and tables of a database according to the present invention.
3 is an exemplary diagram of a matching process.

Hereinafter, a method of linking natural language words of the present invention with columns and tables of a database will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention are provided to more completely explain the present invention to those of ordinary skill in the art, and the embodiments described below may be modified in various other forms, The scope is not limited to the following examples. Rather, these embodiments are provided so as to more fully and complete the present invention, and to fully convey the spirit of the present invention to those skilled in the art.

The terms used herein are used to describe specific embodiments, not to limit the present invention. As used herein, the singular form may include the plural form unless the context clearly dictates otherwise. Also, as used herein, “comprise” and/or “comprising” refers to the presence of the recited shapes, numbers, steps, actions, members, elements, and/or groups thereof. and does not exclude the presence or addition of one or more other shapes, numbers, movements, members, elements and/or groups. As used herein, the term “and/or” includes any one and any combination of one or more of those listed items.

Although the terms first, second, etc. are used herein to describe various members, regions and/or regions, it is to be understood that these elements, parts, regions, layers and/or regions are not limited by these terms. . These terms do not imply a specific order, upper and lower, or superiority, and are used only to distinguish one member, region or region from another. Accordingly, a first member, region, or region described below may refer to a second member, region, or region without departing from the teachings of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to drawings schematically illustrating embodiments of the present invention. In the drawings, variations of the illustrated shape can be expected, for example depending on manufacturing technology and/or tolerances. Therefore, the embodiment of the present invention should not be construed as limited to the specific shape of the region shown in the present specification, but should include, for example, a change in shape caused by manufacturing.

In addition, the present invention relates to a method of linking (matching) columns and tables with natural language words in a method of translating (converting) natural language into SQL queries, and the operating subject of this method is a controller and a storage device capable of normal operation, and A computing device including a memory capable of temporarily storing data, for example, a personal computer or a server may be used.

2 is a flowchart of a method of associating natural language words with columns and tables of a database according to the present invention, and FIG. 3 is an exemplary diagram of a matching process.

2 and 3, the method of linking natural language words with columns and tables of a database according to the present invention includes extracting headwords from natural language sentences and determining words to match (S21), the words and columns or tables Comparing the name with the name (S22), if the result of the determination in step S22 is the same, matching the word with the column or table (S26), and if the result of the determination in step S22 is not the same, the word is the column Alternatively, determining whether the name of the table is partially identical to the name of the table, and if partially identical, performing step S26 (S23), and if not identical as a result of the determination in step S23, checking whether the corresponding word matches a value in the database ( S24) and the step S27 of not linking the word, column and table if the determination result of step S24 does not match, and if the determination result of step S24 matches, check the part-of-speech of the word and if it is a word used in the database Step S26 is performed, and if the word is not used, step S27 is performed (S25).

Hereinafter, the structure and operation of the method of linking the natural language words of the present invention configured as described above to the columns and tables of the database will be described in more detail.

First, as in step S21, a word to be matched is determined by extracting a headword from a natural language sentence.

In this case, the natural language sentence may be input in various ways, such as text input, voice input, or cursive input, and may be a result of extracting text from image input.

In Figure 3, the natural language sentence "Show the names of republican people, and how many times they were at negative side?" is taken as an example, of which names, republican, people, and negative are extracted as headwords, and words to be matched is shown as an example of selection.

Next, in step S22, the word selected in step S21 is compared with the column or table name of the database to determine whether the same column or table exists.

As a result of the determination in step S22, if a specific word is the same as a column or table name, as in step S26, a column or table with the same name as the corresponding word is connected to each other.

3 shows a connection relationship between a word selected in a natural language sentence and a database.

More specifically, the word 'people' selected in the natural language sentence matches the table name 'people' in the database, thus linking the word and the table to each other.

In this comparison, all selected words and all columns and tables of the database are sequentially compared. For the word people, the table with the table name people is connected.

Also, the word 'negative' selected in the natural language sentence matches 'negative', which is one of the columns of the table whose table name is debate_people, and thus the word 'negative' and the column 'negative' are connected to each other.

Next, if the result of the determination in step S22 is not the same, it is determined whether the corresponding word is partially identical to the name of a column or table.

At this time, it is not determined whether the words 'people' and 'negative', which are already connected to a column or table, are partially identical again.

If there is a partially identical column or table as a result of the determination in step S22, step S26 is performed to match the partially identical column or table with a specific word.

In FIG. 3, the word 'names' is partially identical to the column 'name' of the table whose name is people, and thus the word 'names' matches the column 'name' by performing step S26.

Then, if the determination result in step S23 is not the same, it is checked whether the corresponding word matches the value in the database as in step S24.

In this case, it is not checked whether the word matches the value in the database for the previously matched word by the determination of identity and partial identity.

That is, 'people', 'negative' and 'names' are not checked against values in the database.

If the determination result in step S24 does not match, the word, column, and table are not connected and the process is terminated.

Conversely, if a specific word matches the internal value of the database as a result of the determination in step S24, the part-of-speech of the corresponding word is checked as in step S25, and if it is a word used in the database, step S26 is performed. do.

In step S25, words that are not suitable as values in the database are defined in advance so that they are not connected even if the values match.

For example, values used in the database such as 'determiner' and 'preposition' have no meaning even if they match words extracted from natural language sentences.

The word 'republican' extracted from the natural language sentence in FIG. 3 is a data value of the 'party' column of the table named 'people', and thus the word 'republican' and the database value 'republican' are interconnected.

Through such a process, the present invention can comprehensively determine the correlation between the names of columns and tables and the values of the database for the words extracted from the natural language sentences, avoid matching multiple words, and even when only a portion of the words are matched There is a feature that can interconnect words and tables or columns.

In addition, by preventing the word from being matched in multiple columns or tables, natural language words can accurately find meaning.

It is obvious to those of ordinary skill in the art that the present invention is not limited to the above embodiments and can be variously modified and modified within the scope without departing from the technical gist of the present invention. will be.

Claims (6)

A method of matching words in a natural language sentence with columns and tables of a database using a computing device including a microcomputer, the method comprising:
a) checking whether natural language words match a column or table name of a database, and matching the matching column or table to the corresponding word;
b) if there are words that do not match in step a), checking whether the column or table name partially matches the word, and matching the partially matched words; and
c) If there is a natural language word that does not even partially match in step b), check whether it matches the values in the database, match it if it matches, and exclude it from matching if it does not match the database column and how to associate it with a table. According to claim 1,
In step a), the natural language words are
A method of linking natural language words extracted from natural language sentences through headword extraction with columns and tables of a database. According to claim 1,
The word matched with the column or table in step a) is,
A method of linking natural language words with columns and tables of a database, characterized in that the partial identity determination of step b) is not performed. According to claim 1,
The word matched with the column or table in step a) and step b) is,
A method of linking natural language words with columns and tables of a database, characterized in that the comparison with the database internal values of step c) is not performed. According to claim 1,
Step c) is,
c-1) a process of determining whether a word matches an internal value of the database; and
c-2) When the result of the c-1) process matches, if the corresponding word is a word that is not suitable as a predefined database internal value, a natural language word including a non-matching process is linked with the columns and tables of the database. Way. 6. The method of claim 5,
The c-2) process is
A method of linking natural language words with columns and tables in a database, characterized in that it is determined whether a word is an inappropriate word with a predefined database internal value through part-of-speech analysis of a word that matches an internal value in the database.

Images