CN114328951B - Knowledge graph construction method integrating information acquisition and triplet extraction - Google Patents
Knowledge graph construction method integrating information acquisition and triplet extraction Download PDFInfo
- Publication number
- CN114328951B CN114328951B CN202111538747.9A CN202111538747A CN114328951B CN 114328951 B CN114328951 B CN 114328951B CN 202111538747 A CN202111538747 A CN 202111538747A CN 114328951 B CN114328951 B CN 114328951B
- Authority
- CN
- China
- Prior art keywords
- news
- knowledge graph
- triplet
- subject
- extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 39
- 238000010276 construction Methods 0.000 title claims abstract description 22
- 230000004927 fusion Effects 0.000 claims abstract description 11
- 238000005516 engineering process Methods 0.000 claims abstract description 9
- 238000003058 natural language processing Methods 0.000 claims abstract description 9
- 230000009193 crawling Effects 0.000 claims abstract description 8
- 238000012800 visualization Methods 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 27
- 238000004458 analytical method Methods 0.000 claims description 24
- 238000002372 labelling Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 5
- 230000001419 dependent effect Effects 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 3
- 230000008520 organization Effects 0.000 abstract description 4
- 238000005553 drilling Methods 0.000 description 10
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
Landscapes
- Machine Translation (AREA)
Abstract
The invention relates to a knowledge graph construction method for fusion information acquisition and triplet extraction, which comprises the following steps: s1: periodically crawling marine related text contents including news from a specified webpage by utilizing a crawler technology; s2: performing entity extraction and relation extraction on the text content by using a natural language processing tool to obtain a news triplet, and then storing the news triplet into a database; s3: constructing a knowledge graph according to the triples in the database, and realizing the visualization of the knowledge graph in a data browser; s4: and acquiring the association of the knowledge according to the visualized knowledge graph. In the scheme, information acquisition and triplet extraction are fused to construct a knowledge graph, and the whole flow is constructed into an end-to-end task, so that the use cost of a user is reduced; the ocean knowledge graph is constructed, knowledge is found out from scattered data, the relevance of things is mined, and the organization is helped to make guiding decisions.
Description
Technical Field
The invention relates to the field, in particular to a knowledge graph construction method for fusion information acquisition and triplet extraction.
Background
With the continued advancement of global integration, the better and faster development of the marine industry has been the national primary strategic goal. The marine industry is a broad field, including industrial directions of marine fishery, marine economy, marine military, marine environmental protection, and the like. Because of certain relation among industries and certain difference among the characteristics of the industries, how to realize information communication and linkage utilization among the industries becomes a bottleneck for developing the marine industry. Along with the development of technology, the knowledge graph is used as a tool for revealing the dynamic development rule of the knowledge field and providing a practical and valuable reference for disciplinary research, and gradually enters the field of vision of people. At present, the work on the knowledge base of the marine industry is still basically blank due to the professional information, so that the construction of a large-scale semantic knowledge base of the emerging marine industry becomes a requirement.
Knowledge graph is a semantic representation of real world, wherein each node represents an entity, the edges connecting the nodes correspond to the relationship between the entities, and heterogeneous data is expressed as knowledge through integration, and the knowledge is information corresponding to points or edges. Knowledge maps organize all entities with relationships to form a directed graph structure, and the representation of the graph maps the human cognitive way to the world. The knowledge graph displays the complex knowledge field and knowledge system through data mining, information processing, knowledge metering and graphic drawing, reveals the development dynamics and rules of the field, provides the reference of omnibearing, integral and relation chain for the research of the field, and is important production data of the intelligent society. The traditional research direction is mainly focused on constructing a knowledge graph, such as knowledge updating, knowledge fusion and the like, and in specific practice, data acquisition and data cleaning are required to be considered besides the final construction of the knowledge graph, because the quality of the final knowledge graph is deeply influenced by the two steps.
The prior art discloses a system and a method for promoting structured knowledge graph construction by utilizing group learning behaviors, wherein the system comprises an information extraction module, an individual knowledge graph construction module, a group knowledge graph construction module, a knowledge graph construction module and a feedback module. The invention is suitable for middle and primary school teaching and online learning, and a learner establishes the association of a knowledge point Ki and a topic knowledge point set K { K1, K2, K3 … Ki … Kt } to which the Ki belongs through the operation of the related content of the knowledge point Ki in the learning process of the knowledge point Ki to form an individual knowledge graph, wherein the knowledge association is expressed in the form of triples < acceptance, relationship and property >; the individual knowledge patterns of a plurality of learners are aggregated to form a group knowledge pattern Gc; performing reliability analysis on all triples in the group knowledge graph, removing triples which do not reach a reliability threshold value, obtaining a knowledge point graph Gk authoritative by a knowledge point Ki, and perfecting the formed knowledge point graph by an expert; the final knowledge point map can provide feedback for the study of a learner, but the information cannot be communicated and utilized in a linkage way in the aspect of information acquisition.
Disclosure of Invention
The invention provides a knowledge graph construction method for fusion information acquisition and triplet extraction, which aims to solve the technical defects that how to realize information penetration and linkage utilization among various existing industries, knowledge cannot be found in scattered data, and the use cost of a user is increased.
In order to achieve the aim of the invention, the technical scheme adopted is as follows:
a knowledge graph construction method integrating information acquisition and triplet extraction comprises the following steps:
S1: periodically crawling marine related text contents including news from a specified webpage by utilizing a crawler technology;
S2: performing entity extraction and relation extraction on the text content by using a natural language processing tool to obtain a news triplet, and then storing the news triplet into a database;
S3: constructing a knowledge graph according to the triples of news in the database, and realizing the visualization of the knowledge graph in a data browser;
S4: and acquiring the association of the knowledge according to the visualized knowledge graph.
In the scheme, information acquisition and triplet extraction are fused to construct a knowledge graph, and the whole flow is constructed into an end-to-end task, so that the use cost of a user is reduced; the ocean knowledge graph is constructed, knowledge is found out from scattered data, the relevance of things is mined, and the organization is helped to make guiding decisions.
Preferably, step S1 comprises the steps of:
S11: setting a target crawler address, simply analyzing html language of a website, and completing a crawler frame according to a template to obtain news;
In the above scheme, the following fields are parsed and stored: news headlines (news_title), news types (news_type), news dates (news_date), news summaries (news_summary), news bodies (news_content), news web pages (news_web_url), and news sources (news_source).
The database news table fields are as follows:
S12: searching in a database according to the URL of the news as the identifier so as to judge whether the news is repeated or not;
s13: in order to simplify the user operation, a timing function is set so that the crawler program periodically crawls news of a specified website according to the setting of the user.
Preferably, in step S12, the initial dataset is constructed and the news of the designated type and the start and end pages thereof are crawled to the repeated news, and the news is skipped and the next news is continued; when the repeated news is crawled, the secondary website of the news type is crawled, and then the next secondary website of the news type is crawled.
Preferably, step S2 comprises the steps of:
s21: constructing a relation triplet data table, analyzing and storing the fields;
In the above scheme, the following fields are parsed and stored: index (id), subject (triple_subject), subject tag (triple_subject_label), verb (triple_verb), object (triple_subject), object tag (triple_subject_label), original (original_text), table (original_text_table) in which the original is located, and index (original_text_table_id) in which the original is located.
The database triplet relationship table fields are as follows:
S22: when reading the data in the database obtained in the information acquisition stage, firstly reading the data in the news headline field into a cache, and marking the parts of speech of the news headline read into the cache;
S23: carrying out syntactic analysis on sentences of news, and maintaining a dictionary for storing syntactic dependency child nodes for each word in the sentences;
S24: carrying out semantic analysis on sentences by utilizing the results of S21 and S22 and labeling semantic roles;
s25: performing triplet extraction by means of the results of S21, S22 and S23;
in the above scheme, if the semantic role flag is empty, the dependency syntax is used to extract predicate-centric fact triples: mainly processes the three conditions, namely, a main guest; post-fixed language, moving guest relation; the main and auxiliary relationships contain the meson relationship.
S26: screening out repeated triples, removing meaningful punctuation marks by using a regular expression, writing the triples into a triples table, and storing the triples of news into a database.
Preferably, in step S22, it is checked whether the data has been extracted before the triplet extraction, and if so, skipped; if not, word segmentation is carried out by using a natural language processing platform, and word segmentation is a process of recombining continuous word sequences into word sequences according to a certain specification;
Part-of-speech tagging is a text data processing technique in corpus linguistics that tags parts-of-speech of words in the corpus according to their meaning and contextual content.
Preferably, in step S23, the sentence is parsed to maintain a dictionary of syntactically dependent child nodes for each word in the sentence;
The syntactic analysis refers to analyzing the grammar function of words in sentences, is a direct realization of the idea of language block analysis, and simplifies the description of sentences by identifying high-level structural units; and (5) using the results of S21 and S22 to find child nodes of the sentence.
Preferably, in step S24, the semantic role labeling is a shallow semantic analysis technology, and the predicate-argument structure of the sentence is analyzed by taking the sentence as a unit, so that the semantic information contained in the sentence is not deeply analyzed; the task of semantic role labeling is to study the relationship between each component in a sentence and predicates centered on the predicates of the sentence, and describe the relationship between them with semantic roles.
Preferably, step S3 comprises the steps of:
s31: constructing a triplet data table according to triples in the database, sequentially acquiring triplet relations from the triplet data table, and acquiring subjects, subject labels, predicates, objects and object labels in the triplet relations;
S32: searching whether the subject and the object exist in the knowledge graph by using a map query language Cypher of Neo4 j; if the same subject exists, inquiring whether the same subject label exists; if the same object exists, inquiring whether the same object tag exists;
S33: using a Cypher statement to try to insert subjects, subject labels, predicates, objects and object labels into the knowledge graph, and if the knowledge graph is unsuccessful, judging that the relationship does not exist;
s34: and adding new subject labels and object labels for the relations, creating new relations, and after writing all the triplet relations, starting neo4j service by a user and entering a visual interface of the knowledge graph.
Preferably, in step S32, if the same subject exists, it is queried whether the same subject tag exists: if the same subject label does not exist, a new subject label is added for the subject; if the same subject labels exist, the next step is carried out; if the same subject does not exist, a new node is created on the knowledge graph;
If the same object exists, then query whether the same object tag exists: if the same object label does not exist, a new object label is added for the subject; if the same object tag exists, the process advances to step S33. If the same object does not exist, a new node is created on the knowledge graph.
Preferably, in step S4, the knowledge in the knowledge graph is displayed in the relationship of the triples, which includes a subject, a subject tag, a predicate, an object, and an object tag, and a knowledge base is used to query a certain element alone, obtain information of other elements associated with the certain element according to the element, and if overlapping elements exist among the triples, indicate that a certain association exists among the triples.
Compared with the prior art, the invention has the beneficial effects that:
According to the knowledge graph construction method integrating information acquisition and triplet extraction, the knowledge graph is constructed by integrating the information acquisition and triplet extraction, the whole process is constructed into an end-to-end task, and the use cost of a user is reduced; the ocean knowledge graph is constructed, knowledge is found out from scattered data, the relevance of things is mined, and the organization is helped to make guiding decisions.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a flow chart of a method of information acquisition according to the present invention;
FIG. 3 is a flow chart of a method of triad extraction according to the present invention;
FIG. 4 is a flow chart of a method of knowledge graph construction of the present invention;
FIG. 5 is a flow chart of information acquisition according to the present invention;
FIG. 6 is a flowchart of triplet extraction in accordance with the present invention;
FIG. 7 is a flow chart of knowledge graph construction in accordance with the present invention;
FIG. 8 is a diagram showing the effect of the term "Guangdong" search in the knowledge graph according to the present invention;
FIG. 9 is a diagram showing the effect of developing "ocean economic province" in the knowledge graph of the present invention;
FIG. 10 is a diagram showing the effect of developing the "China sea police boat" in the knowledge graph according to the present invention;
fig. 11 is a diagram showing the effect of developing the invention on a "one 20 ten thousand ton class uk ship" within a knowledge graph.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the present patent;
The invention is further illustrated in the following figures and examples.
Example 1
As shown in fig. 1,2, 3 and 4, a knowledge graph construction method for acquiring fusion information and extracting triples includes the following steps:
S1: periodically crawling marine related text contents including news from a specified webpage by utilizing a crawler technology;
S2: performing entity extraction and relation extraction on the text content by using a natural language processing tool to obtain a news triplet, and then storing the news triplet into a database;
S3: constructing a knowledge graph according to the triples of news in the database, and realizing the visualization of the knowledge graph in a data browser;
S4: and acquiring the association of the knowledge according to the visualized knowledge graph.
In the scheme, information acquisition and triplet extraction are fused to construct a knowledge graph, and the whole flow is constructed into an end-to-end task, so that the use cost of a user is reduced; the ocean knowledge graph is constructed, knowledge is found out from scattered data, the relevance of things is mined, and the organization is helped to make guiding decisions.
Preferably, step S1 comprises the steps of:
S11: setting a target crawler address, simply analyzing html language of a website, and completing a crawler frame according to a template to obtain news;
In the above scheme, the following fields are parsed and stored: news headlines (news_title), news types (news_type), news dates (news_date), news summaries (news_summary), news bodies (news_content), news web pages (news_web_url), and news sources (news_source).
The database news table fields are as follows:
S12: searching in a database according to the URL of the news as the identifier so as to judge whether the news is repeated or not;
s13: in order to simplify the user operation, a timing function is set so that the crawler program periodically crawls news of a specified website according to the setting of the user.
Preferably, in step S12, the initial dataset is constructed and the news of the designated type and the start and end pages thereof are crawled to the repeated news, and the news is skipped and the next news is continued; when the repeated news is crawled, the secondary website of the news type is crawled, and then the next secondary website of the news type is crawled.
Preferably, step S2 comprises the steps of:
s21: constructing a relation triplet data table, analyzing and storing the fields;
In the above scheme, the following fields are parsed and stored: index (id), subject (triple_subject), subject tag (triple_subject_label), verb (triple_verb), object (triple_subject), object tag (triple_subject_label), original (original_text), table (original_text_table) in which the original is located, and index (original_text_table_id) in which the original is located.
The database triplet relationship table fields are as follows:
S22: when reading the data in the database obtained in the information acquisition stage, firstly reading the data in the news headline field into a cache, and marking the parts of speech of the news headline read into the cache;
S23: carrying out syntactic analysis on sentences of news, and maintaining a dictionary for storing syntactic dependency child nodes for each word in the sentences;
S24: carrying out semantic analysis on sentences by utilizing the results of S21 and S22 and labeling semantic roles;
s25: performing triplet extraction by means of the results of S21, S22 and S23;
in the above scheme, if the semantic role flag is empty, the dependency syntax is used to extract predicate-centric fact triples: mainly processes the three conditions, namely, a main guest; post-fixed language, moving guest relation; the main and auxiliary relationships contain the meson relationship.
S26: screening out repeated triples, removing meaningful punctuation marks by using a regular expression, writing the triples into a triples table, and storing the triples of news into a database.
Preferably, in step S22, it is checked whether the data has been extracted before the triplet extraction, and if so, skipped; if not, word segmentation is carried out by using a natural language processing platform, and word segmentation is a process of recombining continuous word sequences into word sequences according to a certain specification;
Part-of-speech tagging is a text data processing technique in corpus linguistics that tags parts-of-speech of words in the corpus according to their meaning and contextual content.
Preferably, in step S23, the sentence is parsed to maintain a dictionary of syntactically dependent child nodes for each word in the sentence;
The syntactic analysis refers to analyzing the grammar function of words in sentences, is a direct realization of the idea of language block analysis, and simplifies the description of sentences by identifying high-level structural units; and (5) using the results of S21 and S22 to find child nodes of the sentence.
Preferably, in step S24, the semantic role labeling is a shallow semantic analysis technology, and the predicate-argument structure of the sentence is analyzed by taking the sentence as a unit, so that the semantic information contained in the sentence is not deeply analyzed; the task of semantic role labeling is to study the relationship between each component in a sentence and predicates centered on the predicates of the sentence, and describe the relationship between them with semantic roles.
Preferably, step S3 comprises the steps of:
s31: constructing a triplet data table according to triples in the database, sequentially acquiring triplet relations from the triplet data table, and acquiring subjects, subject labels, predicates, objects and object labels in the triplet relations;
S32: searching whether the subject and the object exist in the knowledge graph by using a map query language Cypher of Neo4 j; if the same subject exists, inquiring whether the same subject label exists; if the same object exists, inquiring whether the same object tag exists;
S33: using a Cypher statement to try to insert subjects, subject labels, predicates, objects and object labels into the knowledge graph, and if the knowledge graph is unsuccessful, judging that the relationship does not exist;
s34: and adding new subject labels and object labels for the relations, creating new relations, and after writing all the triplet relations, starting neo4j service by a user and entering a visual interface of the knowledge graph.
Preferably, in step S32, if the same subject exists, it is queried whether the same subject tag exists: if the same subject label does not exist, a new subject label is added for the subject; if the same subject labels exist, the next step is carried out; if the same subject does not exist, a new node is created on the knowledge graph;
If the same object exists, then query whether the same object tag exists: if the same object label does not exist, a new object label is added for the subject; if the same object tag exists, the process advances to step S33. If the same object does not exist, a new node is created on the knowledge graph.
Preferably, in step S4, the knowledge in the knowledge graph is displayed in the relationship of the triples, which includes a subject, a subject tag, a predicate, an object, and an object tag, and a knowledge base is used to query a certain element alone, obtain information of other elements associated with the certain element according to the element, and if overlapping elements exist among the triples, indicate that a certain association exists among the triples.
Example 2
As shown in fig. 5, the crawler technology is used to periodically crawl marine related text content including news from a designated web page, which comprises the following steps:
setting a target crawler address, simply analyzing html language of a website, and completing a crawler frame according to a template. For the next processing we will parse and store the following fields: news headlines, news types, news dates, news summaries, news bodies, news web pages, and news sources.
And searching in a database according to the URL of the news as an identifier so as to judge whether the news is repeated. When the news of the appointed type and the start and end pages are crawled to the repeated news, the news can be skipped, and the next news can be continued. Crawling updated news differs from crawling the next type of secondary web site to crawl when repeated news is crawled, indicating that the secondary web site of that news type has been crawled.
Finally, to simplify the user operation, we set a timing function, and the crawler program can periodically crawl news of a specified website according to the user's settings.
Example 3
As shown in fig. 6, entity extraction and relation extraction are performed on text contents by using a natural language processing tool to obtain news and triples, and then the news and triples are stored in a database, which comprises the following steps:
first, construct a relational triplet data table, we will parse and store the following fields: index, subject label, verb, object label, original, table of original, and index of table of original
Secondly, reading the data in the database obtained in the information acquisition stage, firstly, attempting to read the data in the news headline (news_title) field into the cache. Checking whether the data is extracted before the triplet extraction, and skipping if so; if not, the halving natural language processing platform (pyltp) is utilized to perform word segmentation, and word segmentation is a process of recombining continuous word sequences into word sequences according to a certain specification. Such as: news headlines: the offshore drilling work is finished in the first submarine high-speed rail tunnel in China, and the results after word segmentation are as follows: 'domestic', 'first', 'strip', 'subsea', 'high-speed rail', 'tunnel', 'already', 'complete', 'offshore', 'drilling', 'working'.
The news headlines (news_title) that have been read into the cache are then part-of-speech tagged. Part-of-speech tagging is a text data processing technique in linguistic corpus that tags parts-of-speech of words in the corpus according to their meaning and context, such as: news headlines: the offshore drilling work is finished in the first submarine high-speed rail tunnel in China, and the part of speech tagging is carried out by using the result of S1: [ 'nl','m', 'q', 'n', 'n', 'n','d', 'v', 'nl', 'v', 'v' ]. If the one-to-one correspondence with the S1 result is shown as follows: { 'domestic': 'nl', 'first': m ',' strip ',' q ',' subsea ':' n ',' high speed rail ',' tunnel ',' n ',' already-freighted ',' v ',' complete ',' offshore ',' nl ',' drilling ',' working ',' v ','.
Then, the sentence is parsed, and a dictionary of syntactically dependent child nodes is maintained for each word in the sentence. The syntactic analysis refers to the analysis of the grammar function of words in sentences, is a direct realization of the concept of the analysis of language blocks, and simplifies the description of sentences by identifying high-level structural units. With the results of the first two steps, child nodes of the sentence can be found. If the title is continuously utilized, the child node can be obtained as ：[{},{},{'ATT':[1]},{},{'ATT':[3]},{'ATT':[0,2,4]},{},{'SBV':[5],'ADV':[6],'VOB':[10]},{},{'ADV':[8]},{'ATT':[9]}].
And carrying out semantic analysis on the sentences by using the results of the first two steps and labeling semantic roles. Semantic role labeling is a shallow semantic analysis technology, which analyzes predicate-argument structures of sentences by taking sentences as units, and does not carry out deep analysis on semantic information contained in the sentences. Specifically, the task of semantic role labeling is to study the relationship between each component in a sentence and predicates centered on the predicates of the sentence, and describe the relationship between them with semantic roles. If the news headline is continuously utilized, the result after semantic role marking is as follows: {7 { 'A0': [ 'A0',0,5], 'ADV': [ 'ADV', 6], 'A1': [ 'A1',8,10] }; if the news headline corresponds to the news headline, the news headline is: { "complete" { 'A0': [ 'A0', "first seafloor high-speed rail tunnel in China" ], ADV ': [' ADV ',6 "], A1': [ 'A1'," offshore drilling operation "] }.
And performing triplet extraction by means of the results of the first four steps. If the semantic role flag is null, then the dependency syntax is used to extract predicate-centric fact triples: mainly processes the three conditions, namely, a main guest; post-fixed language, moving guest relation; the main and auxiliary relationships contain the meson relationship. If the news headline is continuously utilized, the triple relation can be obtained by the result of the first four steps: the first seafloor high-speed rail tunnel in China, 'complete', 'offshore drilling operations' ].
Finally, the repeated triplet relationships are screened out and the punctuation marks of the meaning are removed by using the regular expression, and then the triples are written into a triplet table. If the title is empty, then all steps are repeated in the abstract.
Example 4
As shown in fig. 7, a knowledge graph is constructed according to triples in a database, and visualization of the knowledge graph is realized in a data browser, which comprises the following steps:
First, a triple relation is sequentially obtained from a triple data table, and subjects, subject labels, predicates, objects and object labels in the triple relation are obtained. For example, if the triplet is [ 'domestic first seafloor high-speed rail tunnel', 'finish', 'offshore drilling operation', 'domestic first seafloor high-speed rail tunnel', subject label is 'noun', predicate is 'finish', object is 'offshore drilling operation', object label is 'event'.
Second, query language cyto with Neo4j tries to find if the subject and object are present in the knowledge-graph. For example, if the result obtained in S1 is used, the subject query term is: match (n) where n.name= "{ 'first seafloor high-speed tunnel in China' }" return n, object query statement is: match (n) where n.name= "{ 'offshore drilling work' }" return n.
Again, if the same subject exists, then query whether the same subject tag exists: if the same subject label does not exist, a new subject label is added for the subject; if the same subject label exists, the next step is entered. If the same subject does not exist, a new node is created on the knowledge graph.
Then, if the same object exists, it is queried whether the same object tag exists: if the same object label does not exist, a new object label is added for the subject; if the same object tag exists, the next step is entered. If the same object does not exist, a new node is created on the knowledge graph.
Then, the Cypher statement is used to try to insert the subject, subject tag, predicate, object and object tag obtained from the first step in the knowledge graph. For example, if the result of the first step is used, the insert sentence is Match (n: 'noun') [: 'complete' ] - > (m: 'event') where n.name= "domestic first seafloor high-speed rail tunnel" and m.name= "offshore drilling work" return m, and then it is checked whether the insert was successful.
If the relationship is unsuccessful, then it can be determined that the relationship does not exist. New subject and object tags are then added to the relationship, and a new relationship is then created.
Finally, after writing all triples, the user may initiate neo4j services. After entering the visual interface, clicking related buttons and typing a Cypher sentence, and finally, carrying out knowledge reasoning based on the query.
Example 5
As shown in fig. 8, 9, 10 and 11, in the knowledge base, knowledge is presented in a triplet relationship, { subject, subject tag, predicate, object tag }. Where a subject refers to an issuer of an action, a predicate refers to a link of the subject with an object, and an object refers to a recipient of the predicate action. One element can be queried independently through the knowledge base, and then other element information associated with the element can be acquired according to the element. If there are coincident elements between multiple triples, then it is stated that there is some correlation between the triples.
Example 1: all elements associated with this element of "Guangdong" are queried. The results show that, firstly, no arrow points to the Guangdong, which means that the input degree is 0, the output degree is the number of elements which are related to the query, the Guangdong is all taken as a subject in a knowledge base, and we can judge that the captured news is mostly described by standing at the angle of the Guangdong. On the basis of fig. 8, we show the element "ocean economic strong province".
In fig. 9, it can be seen that the subject associated with "ocean economic strong province" also has "Fujian Provincial Party committee province government", which can obtain knowledge from a certain degree of correlation, and understand that "Guangdong" and "Fujian Provincial Party committee province government" have similar construction targets.
Example 2: all elements associated with this element "deep sea" are queried. The results of fig. 10 show that it can be seen first that there are arrows pointing to "deep sea" and arrows pointing from "deep sea" to other elements, which illustrates that the relationship of other elements to "deep sea" is multiple.
We point out one of the elements "work" that forms a prime guest relationship with "deep sea", and can see from fig. 10 and 11 that there are subjects such as "chinese sea police boat", "one 20 ten thousand ton class uk ship" that work in deep sea, so we can know from one element "deep sea" other knowledge associated with it.
It is to be understood that the above examples of the present invention are provided by way of illustration only and not by way of limitation of the embodiments of the present invention. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the invention are desired to be protected by the following claims.
Claims (8)
1. The knowledge graph construction method integrating information acquisition and triplet extraction is characterized by comprising the following steps of:
S1: periodically crawling marine related text contents including news from a specified webpage by utilizing a crawler technology;
S2: performing entity extraction and relation extraction on the text content by using a natural language processing tool to obtain a news triplet, and then storing the news triplet into a database;
S3: constructing a knowledge graph according to the triples of news in the database, and realizing the visualization of the knowledge graph in a data browser;
S4: acquiring knowledge association according to the visualized knowledge graph;
Step S2 comprises the steps of:
s21: constructing a relation triplet data table, analyzing and storing the fields;
S22: when reading the data in the database obtained in the information acquisition stage, firstly reading the data in the news headline field into a cache, and marking the parts of speech of the news headline read into the cache;
S23: carrying out syntactic analysis on sentences of news, and maintaining a dictionary for storing syntactic dependency child nodes for each word in the sentences;
S24: carrying out semantic analysis on sentences by utilizing the results of S21 and S22 and labeling semantic roles;
s25: performing triplet extraction by means of the results of S21, S22 and S23;
S26: screening out repeated triplet relation, removing significant punctuation marks by using regular expression, writing the triplet into a triplet table, and storing the triplet of news into a database;
Step S3 comprises the steps of:
s31: constructing a triplet data table according to triples in the database, sequentially acquiring triplet relations from the triplet data table, and acquiring subjects, subject labels, predicates, objects and object labels in the triplet relations;
S32: searching whether the subject and the object exist in the knowledge graph by using a map query language Cypher of Neo4 j; if the same subject exists, inquiring whether the same subject label exists; if the same object exists, inquiring whether the same object tag exists;
S33: using a Cypher statement to try to insert subjects, subject labels, predicates, objects and object labels into the knowledge graph, and if the knowledge graph is unsuccessful, judging that the relationship does not exist;
s34: and adding new subject labels and object labels for the relations, creating new relations, and after writing all the triplet relations, starting neo4j service by a user and entering a visual interface of the knowledge graph.
2. The knowledge graph construction method for fusion information acquisition and triplet extraction according to claim 1, wherein the step S1 comprises the steps of:
S11: setting a target crawler address, simply analyzing html language of a website, and completing a crawler frame according to a template to obtain news;
S12: searching in a database according to the URL of the news as the identifier so as to judge whether the news is repeated or not;
s13: in order to simplify the user operation, a timing function is set so that the crawler program periodically crawls news of a specified website according to the setting of the user.
3. The knowledge graph construction method for fusion information acquisition and triad extraction according to claim 2, wherein in step S12, the construction of the initial data set and crawling of news of a designated type and the start and end pages thereof skip the news and continue with the next news when crawling of repeated news; when the repeated news is crawled, the secondary website of the news type is crawled, and then the next secondary website of the news type is crawled.
4. A method of knowledge graph construction for fusion information acquisition and triplet extraction as claimed in claim 3, wherein in step S22, it is checked whether the data has been extracted before triplet extraction, and if so, skipped; if not, word segmentation is carried out by using a natural language processing platform, and word segmentation is a process of recombining continuous word sequences into word sequences according to a certain specification;
Part-of-speech tagging is a text data processing technique in corpus linguistics that tags parts-of-speech of words in the corpus according to their meaning and contextual content.
5. The method according to claim 4, wherein in step S23, the sentence is subjected to syntactic analysis, and a dictionary for storing syntactically dependent child nodes is maintained for each word in the sentence;
The syntactic analysis refers to analyzing the grammar function of words in sentences, is a direct realization of the idea of language block analysis, and simplifies the description of sentences by identifying high-level structural units; and (5) using the results of S21 and S22 to find child nodes of the sentence.
6. The method for constructing a knowledge graph with fusion information acquisition and triplet extraction as claimed in claim 5, wherein in step S24, semantic role labeling is a shallow semantic analysis technique, and the predicate-argument structure of a sentence is analyzed in sentence units without further analysis of semantic information contained in the sentence; the task of semantic role labeling is to study the relationship between each component in a sentence and predicates centered on the predicates of the sentence, and describe the relationship between them with semantic roles.
7. The method for constructing a knowledge graph with fusion information acquisition and triad extraction according to claim 6, wherein in step S32, if the same subject exists, it is queried whether the same subject label exists: if the same subject label does not exist, a new subject label is added for the subject; if the same subject labels exist, the next step is carried out; if the same subject does not exist, a new node is created on the knowledge graph;
if the same object exists, then query whether the same object tag exists: if the same object label does not exist, a new object label is added for the subject; if the same object tag exists, the process proceeds to step S33; if the same object does not exist, a new node is created on the knowledge graph.
8. The method for constructing a knowledge graph with fusion information acquisition and triplet extraction according to claim 7, wherein in step S4, knowledge in the knowledge graph is displayed in a relation of triples, and the knowledge graph comprises a subject, a subject tag, a predicate, an object and an object tag, wherein a knowledge base is used for independently inquiring a certain element, and other element information associated with the certain element is acquired according to the element, and if overlapping elements exist among a plurality of triples, a certain association exists among the triples.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111538747.9A CN114328951B (en) | 2021-12-15 | 2021-12-15 | Knowledge graph construction method integrating information acquisition and triplet extraction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111538747.9A CN114328951B (en) | 2021-12-15 | 2021-12-15 | Knowledge graph construction method integrating information acquisition and triplet extraction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114328951A CN114328951A (en) | 2022-04-12 |
| CN114328951B true CN114328951B (en) | 2024-08-27 |
Family
ID=81053105
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111538747.9A Active CN114328951B (en) | 2021-12-15 | 2021-12-15 | Knowledge graph construction method integrating information acquisition and triplet extraction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114328951B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115017335A (en) * | 2022-06-16 | 2022-09-06 | 特赞（上海）信息科技有限公司 | Knowledge graph construction method and system |
| CN114880455B (en) * | 2022-07-12 | 2022-12-06 | 科大讯飞股份有限公司 | Triple extraction method, device, equipment and storage medium |
| CN115658913A (en) * | 2022-10-17 | 2023-01-31 | 桂林电子科技大学 | Electric power equipment information map construction method based on Handle identification analysis |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110555153A (en) * | 2019-08-20 | 2019-12-10 | 暨南大学 | Question-answering system based on domain knowledge graph and construction method thereof |
| CN110597999A (en) * | 2019-08-01 | 2019-12-20 | 湖北工业大学 | Judicial case knowledge graph construction method of dependency syntactic analysis relation extraction model |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20000063488A (en) * | 2000-07-18 | 2000-11-06 | 박은일 | The semantic knowledge database automatic construction device on the on-line document, and the method, the record medium for that. |
| CN111897914B (en) * | 2020-07-20 | 2023-09-19 | 杭州叙简科技股份有限公司 | Entity information extraction and knowledge graph construction method for comprehensive pipe rack field |
| CN113486189B (en) * | 2021-06-08 | 2024-10-18 | 广州数说故事信息科技有限公司 | Open knowledge graph mining method and system |
-
2021
- 2021-12-15 CN CN202111538747.9A patent/CN114328951B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110597999A (en) * | 2019-08-01 | 2019-12-20 | 湖北工业大学 | Judicial case knowledge graph construction method of dependency syntactic analysis relation extraction model |
| CN110555153A (en) * | 2019-08-20 | 2019-12-10 | 暨南大学 | Question-answering system based on domain knowledge graph and construction method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114328951A (en) | 2022-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114328951B (en) | Knowledge graph construction method integrating information acquisition and triplet extraction | |
| CN110399457B (en) | Intelligent question answering method and system | |
| CN106874378B (en) | Method for constructing knowledge graph based on entity extraction and relation mining of rule model | |
| CN110334178B (en) | Data retrieval method, device, equipment and readable storage medium | |
| US7689557B2 (en) | System and method of textual information analytics | |
| CN110110054A (en) | A method of obtaining question and answer pair in the slave non-structured text based on deep learning | |
| CN113822067A (en) | Key information extraction method and device, computer equipment and storage medium | |
| CN113312922B (en) | Improved chapter-level triple information extraction method | |
| CN103064956A (en) | Method, computing system and computer-readable storage media for searching electric contents | |
| CN107918644B (en) | News topic analysis method and implementation system in reputation management framework | |
| CN110609998A (en) | Data extraction method of electronic document information, electronic equipment and storage medium | |
| CN112749265B (en) | Intelligent question-answering system based on multiple information sources | |
| CN112163097A (en) | Military knowledge graph construction method and system | |
| CN113468304A (en) | Construction method of ship berthing knowledge question-answering query system based on knowledge graph | |
| CN110765277A (en) | Online equipment fault diagnosis platform of mobile terminal based on knowledge graph | |
| CN104268283A (en) | Method for automatically analyzing Internet web page | |
| CN115759037A (en) | Intelligent auditing frame and auditing method for building construction scheme | |
| CN112015907A (en) | Method and device for quickly constructing discipline knowledge graph and storage medium | |
| CN103020286B (en) | Based on the internet ranking list grasping system of rank website | |
| CN101350019B (en) | Method for abstracting web page information based on vector model between predefined slots | |
| CN106372232B (en) | Information mining method and device based on artificial intelligence | |
| CN112711666B (en) | Futures label extraction method and device | |
| CN107145591A (en) | Title-based webpage effective metadata content extraction method | |
| CN110750632B (en) | Improved Chinese ALICE intelligent question-answering method and system | |
| CN112115269A (en) | Webpage automatic classification method based on crawler |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |