CN112199372A - Mapping relation matching method and device and computer readable medium - Google Patents
Mapping relation matching method and device and computer readable medium Download PDFInfo
- Publication number
- CN112199372A CN112199372A CN202011017685.2A CN202011017685A CN112199372A CN 112199372 A CN112199372 A CN 112199372A CN 202011017685 A CN202011017685 A CN 202011017685A CN 112199372 A CN112199372 A CN 112199372A
- Authority
- CN
- China
- Prior art keywords
- matching
- mapping
- training
- mapping relation
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013507 mapping Methods 0.000 title claims abstract description 137
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000012549 training Methods 0.000 claims abstract description 80
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 67
- 238000013528 artificial neural network Methods 0.000 claims abstract description 59
- 238000003860 storage Methods 0.000 claims description 34
- 230000008569 process Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 11
- 238000000586 desensitisation Methods 0.000 claims 1
- 230000003993 interaction Effects 0.000 abstract description 14
- 230000001376 precipitating effect Effects 0.000 abstract description 4
- 230000007547 defect Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 16
- 238000012545 processing Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000012937 correction Methods 0.000 description 4
- 238000013506 data mapping Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 241000282326 Felis catus Species 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 238000013499 data model Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 241000282840 Vicugna vicugna Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013501 data transformation Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a mapping relation matching method and device and a computer readable medium. The mapping relation matching method based on artificial intelligence comprises the following steps: inputting the source field and the target field into a neural network via a human-computer interaction interface; calculating the matching degree of the target field and the source field through the neural network to obtain a mapping result table containing the matching degree; adjusting the mapping relation in the mapping result table by a user to obtain a mapping result table containing the adjusted mapping relation; and training the neural network according to the adjusted mapping relation so as to calculate the matching degree through the trained neural network. The invention can overcome the defect that the traditional simple word spelling matching can not be completed, and realizes the function of precipitating experience of an expert system.
Description
Technical Field
The invention relates to a mapping relation matching method, in particular to a mapping relation matching method based on artificial intelligence. In addition, the invention also relates to an electronic device and a computer readable medium applying the matching method.
Background
In conventional program development techniques, data processing and management may be involved. The data processing is the collection, storage, retrieval, processing, transformation and transmission of data. Data is a form of expression for facts, concepts, or instructions that may be processed by human or automated means.
The data may be in the form of numbers, text, graphics, or sound, etc. After the data is interpreted and given a certain meaning, it becomes information. The basic purpose of data processing is to extract and derive valuable, meaningful data for certain people from large, possibly chaotic, unintelligible amounts of data.
Data processing is the basic link of system engineering and automatic control. Data processing is throughout various fields of social production and social life. The development of data processing technology and the breadth and depth of its application have greatly influenced the progress of human society development.
Data management is the process of efficiently collecting, storing, processing, and applying data using computer hardware and software techniques. The purpose of this is to fully and effectively play the role of data. The key to achieving efficient management of data is data organization. With the development of computer technology, data management goes through three development stages of manual management, file systems and database systems.
When the system is faced with different service systems, different component modules and different program foreground and background, a large amount of information needs to be exchanged and transferred, fields need to be screened and matched in a manual mode, the work is heavy, and the error rate is high. The field definition rules in different systems are not uniform, the definition modes have various patterns, the traditional mapping relation matching mode can only be matched after strictly matching names or a fixed conversion mode (such as case conversion), the practical value is not too much, and a manual mode is basically adopted.
The traditional mapping relation matching is based on judging whether the character strings are consistent after being converted by a computer program, the program is directly operated, and a training process is not needed. The most common mapping matching method steps are, for example, unifying naming conventions by unifying case, e.g., all converting to small hump form. The vicuna-Case nomenclature is a set of naming rules (conventions) that are written in computer programs. The vicuna nomenclature is that when the variable or function name is a unique identifying word formed by one or more words joined together, the first word starts with a lower case letter; the capital letter of the second word, or the capital letter of each word, is in capital letters.
Small hump method
Variables are generally identified by the small hump method. Hump means: except for the first word, the first letters of the other words are capitalized. For example int studentCount ═ 0. The first word of the variable myStudentCount is all lowercase, and the first letter of the following word is uppercase.
Great hump method
Compared with the small hump method, the large hump method capitalizes the first letter of the first word. It is commonly used in class name, function name, attribute and name space. Such as a public class student information.
As for the mapping relationship matching method, for example:
the database is named MY _ TABLE _ ID
The Java program is named MyTableID
Json format is my-table-id
After the naming specification is unified through the unified case, for example, the naming specification is unified into a small hump mode, then the result is:
myTableId；
at this time, if the exact matches are consistent, the mapping relationship between the above character strings in different formats is established.
However, there are various cases where the design rule, naming convention, or developer style are different in the use of a program system. If the myTableId in the above example is mapped to the id, the mapping relation needs to be manually judged and established, and the following mapping cannot be automatically realized through a program:
myTableid- -corresponding- -id.
In this case, the problem of the large number of fields between systems and the heavy and mechanically tedious manual completion work for establishing the mapping is encountered. Meanwhile, the method also has the negative effects that the manual matching error rate is high, and the positioning cost is increased sharply when the matching quantity is large.
Disclosure of Invention
In view of the above, a new mapping relationship matching method is needed.
In order to achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a mapping relationship matching method based on artificial intelligence, including: inputting, via an input device, a source field and a target field into a neural network; calculating the matching degree of the target field and the source field through the neural network to obtain a mapping result table containing the matching degree; adjusting the mapping relation in the mapping result table by a user to obtain a mapping result table containing the adjusted mapping relation; and training the neural network according to the adjusted mapping relation so as to calculate the matching degree through the trained neural network.
Optionally, in the adjustment of the mapping relationship, the mapping relationship that needs to be adjusted is determined by comparing the matching degree with a preset safety threshold.
Optionally, in the adjusting of the mapping relationship, the adjusted mapping relationship is stored.
Optionally, the training is performed iteratively in an iterative manner.
Optionally, the matching degree is calculated according to semantics of the source field and the target field.
Optionally, the safety threshold is 80% or greater.
Optionally, the source field and the target field are input via a human-machine interface as the input device.
Optionally, the training is performed by an artificial intelligence training machine.
Optionally, the mapping relation to be adjusted is stored as a sample for training, and training is performed on the sample.
Optionally, the neural network is a CPU or GPU implementing the computation.
Optionally, the training of the neural network is to train an intelligent matching model by an artificial intelligence training computer, and receive a new trained matching model by an artificial intelligence execution machine separate from the artificial intelligence training computer, and calculate the degree of matching using the trained matching model.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided an apparatus for performing mapping relationship matching using an artificial intelligence neural network technique. The device comprises: the system comprises an input module, an artificial intelligence matching module, a storage module, an artificial intelligence training module and a display module, wherein the input module is used for receiving a source field and a target field; the artificial intelligence matching module comprises a neural network, a source field and a target field, and is used for receiving the source field and the target field, and calculating the matching degree of the target field and the source field to obtain a mapping result table containing the matching degree; the storage module is used for storing a mapping result table which is obtained by adjusting the mapping relation in the mapping result table through a user and contains the adjusted mapping relation; the display module is used for displaying the mapping result table of the adjusted mapping relation; and the artificial intelligence training module is used for training the neural network according to the adjusted mapping relation so as to calculate the matching degree through the trained neural network.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided an electronic device for performing mapping relationship matching using an artificial intelligence neural network technique. The electronic device includes: one or more processors; a storage device to store one or more programs. When executed by the one or more processors, cause the one or more processors to implement the methods according to the above-described embodiments of the present invention.
Optionally, in the mapping relation matching apparatus, the artificial intelligence training module is further configured to: in the adjustment of the mapping relation, the matching degree is compared with a preset safety threshold value, so that the mapping relation needing to be adjusted is determined.
Optionally, in the mapping relation matching apparatus, the artificial intelligence training module is further configured to repeatedly perform the training process in an iterative manner.
Optionally, in the mapping relation matching apparatus, the neural network performs calculation of the matching degree according to semantics of a source field and a target field.
Optionally, in the mapping relation matching apparatus, the safety threshold is 80% or more.
Optionally, in the mapping relation matching apparatus, the storage module is further configured to: and storing the mapping relation to be adjusted as a sample for training, and training the sample.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided a computer-readable storage medium. A computer-readable storage medium of an embodiment of the present invention stores thereon a computer program, which, when executed by a processor, implements a method of mapping relationship matching of embodiments of the present invention.
According to the invention, the mapping relation matching is carried out by using the artificial intelligent neural network technology, so that the operation of the correlation degree of the incidence relation of the two fields can be realized by training the neural network, and the function which cannot be completed by the traditional simple word spelling matching is realized. Moreover, the function of precipitating experience of an expert system can be further realized by recording the mapping relation which is considered by the user to be incorrect in matching and using the information as a sample for training a computer to train the neural network again.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of a map matching system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a mapping matching method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a mapping matching apparatus according to an embodiment of the present invention;
fig. 4 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments of the present invention and the technical features of the embodiments may be combined with each other without conflict.
Data mapping is the process of creating two different data models and defining links between the models. The data model may include metadata, i.e., data atomic units that have semantically precise meaning, that use an atomic unit system to measure attributes of electricity containing information.
The primary uses of data mapping include a variety of platforms. Data transformation is a process used to mediate the relationship between an initial data source and a target that uses the data. It is useful in identifying portions of data lineage analysis, the way data flows from one information department to another. Put in place when data mapping is performed. This allows the user to create information or convert information into a form that can be filtered out of the best results. Typically, this takes the form of some graphical mapping tool. The user can literally "draw" a line from field to determine the correct connection. This is called manual data mapping. The data elements themselves need to be identified and named, the explicit definition of the data needs to be determined, and the representation of the values enumerated. In some cases, the identifier is represented in the form of a database. The standard structure is constructed using basic information units such as name, address or age.
According to the invention, two groups of fields, namely the target field and the source field, input by a user are corresponding, for example, Myname corresponds to name, AccountId corresponds to id, and Age corresponds to Age, and then a corresponding relation table is output to a specified position for subsequent use. The operation is carried out through the artificial intelligence neural network trained in advance to generate the mapping relation among the fields, so that efficient mapping matching can be achieved, and the matching result with low error rate can be achieved at lower cost.
FIG. 1 illustrates a mapping matching system according to an embodiment of the invention. The system comprises a human-computer interaction interface unit 101, an artificial intelligence matching unit 103, a mapping result storage unit 108 and an artificial intelligence training calculation unit 109. Further, optionally, a mapping result record display unit, a predetermined threshold input setting unit, and the like may also be included.
The human-machine interface 101 is a display and input device including Web or Windows, for example. The human-machine interface 101 is an integration of the exemplary input and display devices. Alternatively, the input device and the display device may be separately provided as components in the human-computer interaction system. The human-computer interaction system can be a variety of machines, and can also be a computerized system and software. The human-computer interaction interface generally refers to a portion visible to a user. And the user communicates with the system through a human-computer interaction interface and performs operation. Referring to fig. 1, through the human-computer interface 101, a user inputs a source field and a target field 102 through a Web or Windows window interface. For example,
a source field: MyName, AccountId, Age, Base, ComSize
A target field: name, id, age, location, company size
The artificial intelligence matching unit 103 may be constituted by a computer device. In the matching unit 103, a main program and/or a pre-trained neural network are installed. As shown in fig. 1, in the artificial intelligence calculation unit 103, a unit (or component) 104 for performing an artificial intelligence matching operation is provided. The main program is responsible for accepting user input, and further inputting the source field and the target field input by the human-computer interaction interface unit 101 into the neural network constructed in the matching unit. The neural network may calculate the degree of acquaintance (i.e., the degree of match) of the target field with the source field. Regarding the degree of acquaintance (or similarity), it can be understood that, for example, a picture is input in the conventional image recognition, and an Artificial Intelligence (AI) algorithm judges whether the picture contains a cat, i.e., the degree of acquaintance between the picture and the cat is calculated. When the acquaintance reaches a certain value, the algorithm determines that the picture contains the cat.
Further, the main program records the result with the highest degree of recognition to the final mapping result table 105, and stores the result in the storage unit 108. The storage unit 108 may be a conventional storage device, such as a temporary or non-temporary storage device. Optionally, the mapping result 105 may be fed back to the user via the human-machine interface 101. The main program is operated by using a CPU, and the artificial intelligence neural network can be operated by the CPU or the GPU.
The mapping result contains three parts, namely a source field, a target field and a matching degree. The matching degree is calculated by the neural network, and the final matching result is the matching with the strongest correlation. Examples are:
MyName field matching degree list:
TABLE 1
| Source field | MyName | MyName | MyName | MyName | Age MyName |
| Object field | name | id | age | location | companSize |
| Degree of matching | 98％ | 10％ | 1％ | 3％ | 2％ |
As can be seen from Table 1, the MyName field eventually matches 98%.
Name acquaintance list:
TABLE 2
| Source field | Age | Age | Age | Age | Age |
| Object field | name | id | age | location | companSize |
| Degree of matching | 1％ | 1％ | 100％ | 1％ | 2％ |
As can be seen from Table 2, Age finally matches the results: 100 percent.
ComSize acquaintance List:
TABLE 3
| Source field | ComSize | ComSize | ComSize | ComSize | ComSize |
| Object field | name | id | age | location | companSize |
| Degree of matching | 2％ | 3％ | 1％ | 5％ | 10％ |
As can be seen from Table 3, the final match result for ComSize is: 10 percent.
Further, a completion result is determined through a human-computer interaction interface. The user checks the degree of matching. Here, a safety threshold is preset manually or by machine, and the threshold may be, for example, 75%, 80%, 85%, 90%, or 95% or more. Here, a matching degree safety threshold of 80% is set in advance. Fields that need to be determined manually are filtered out by comparison to a safety threshold. When the matching degree is higher than the safety threshold value, the artificial intelligence matching is considered to meet the requirement, the matching is passed, if the matching degree is lower than the safety threshold value, the artificial intelligence confidence is considered to be insufficient, and further manual auxiliary judgment is needed. If the result needs to be adjusted, the user adjusts the mapping relation through a human-computer interaction interface or other input devices, then the mapping result 107 is stored in a storage device, and meanwhile the adjusted field related mapping relation is sent to the artificial intelligence training computer. Optionally, the mapping results 107 before and/or after the auxiliary judgment and the adjustment are stored after the manual determination by the user.
The artificial intelligence training calculation unit 109 may be formed by a computer, and is generally implemented by a GPU supporting hardware acceleration. The neural network is trained by an artificial intelligence training computation unit 109.
The result of the first training may be sent to the artificial intelligence matching calculation unit 103 as an execution machine (execution unit). At a later stage, for example in an iterative manner, the manually-intervened mapping sample may be continuously subjected to repeated correction training to expand the training set. Then, the trained new version neural network is sent to the execution machine 103 for mapping matching operation.
Note that the matching results achieved by the artificial intelligence matching unit 104 are mostly available, but there may be a portion that requires human intervention, according to an embodiment of the present invention. For the partial data and results, a correct matching result is obtained through the artificial dry method, and the matching result is input to the artificial intelligence training unit 109 as a training material for training. Thus, the matching degree and the matching result can be optimized.
Optionally, in the embodiment of the present invention, the artificial intelligence matching unit 103 may match the source data and the target data according to a "semantic" principle. According to the embodiment of the invention, the defect that the semantic relevance matching cannot be realized by the traditional mapping relation matching is solved. For example, in conventional map matching techniques, such as Base is associated with location, but conventional methods cannot establish a match. In the embodiment of the invention, the artificial intelligence matching unit 103 wants to perform matching in consideration of the semantic situation in the artificial intelligence matching link, so that a more effective and accurate matching mapping result can be realized.
In addition, in the embodiment of the invention, the function of an expert system can be realized by manually strengthening the preset identified rule into the training rule by the user. For example, prior to intelligent training, the rules are manually determined by the user: his is related to history and com is related to company. By carrying out intelligent training according to the rule, the expected matching degree can be further improved, so that reasonable and effective matching of the abbreviation field is further realized in the artificial intelligence matching link.
As shown in fig. 2, another embodiment according to the present invention is a method (or process) of mapping matching, which mainly includes the following steps (or processes), for example:
step S101: a user inputs a source field and a target field through a Web or Windows window interface;
step S102: and receiving user input, and inputting the source field and the target field into an artificial intelligence neural network, wherein the neural network calculates the acquaintance degree of the target and the source field. The main program of the neural network records the result with the highest identification degree to a final mapping result table, and the main program instructs to store the result to the storage device and feeds the result back to a user through a human-computer interaction interface;
step S103: calculating the matching degree by a neural network, determining the relevance according to the matching degree, and defining the matching result with the strongest relevance as a final matching result;
step S104: determining a final result through a man-machine interaction interface manually, and checking the matching degree by a user;
step S105: training the neural network through an artificial intelligence training computer, sending a first training result to the intelligent computer, and performing repeated correction training through a mapping sample continuously subjected to artificial intervention in a later period to enlarge a training set.
According to the process of the embodiment, the mapping relation matching can be performed by using an artificial intelligent neural network technology, the operation of the correlation degree of the incidence relation of the two fields is realized by training the neural network, and the function which cannot be completed by the traditional simple word spelling matching is realized. And if the mapping relation which is considered by the user to be incorrect is recorded, and the neural network is trained again by taking the information as a sample for training the computer, the function of precipitating experience by an expert system can be further realized.
Specifically, in step S101, the user inputs the source field and the destination field through a human-computer interface provided in the system, for example, through a Web or Windows window interface. For example,
a source field: MyName, AccountId, Age, Base, ComSize
A target field: name, id, age, location, company size
In step (process) 102, the calculation and recording of the degree of identity is implemented using an artificial intelligence computer as an execution machine.
For example, a main program and a pre-trained neural network are installed on the execution machine. The primary program is responsible for accepting user input and then entering the source and target fields into the neural network. The neural network can calculate the degree of acquaintance between the target field and the source field, the main program records the result with the highest degree of acquaintance to a final mapping result table, and the main program stores the result to the storage device and feeds the result back to a user through a human-computer interaction interface.
It should be noted that the main program can be operated by using a CPU, and the artificial intelligence neural network can be operated by using both the CPU and the GPU.
After the artificial intelligence neural network is used for calculation, the output mapping result comprises three parts of mapping results, namely a source field, a target field and a matching degree. Specifically, the matching degree is calculated by the neural network. A safety threshold is preset manually or by machine and may be, for example, 75%, 80%, 85%, 90%, or 95% and above. And comparing the calculated matching degree with a preset safety threshold (for example, 80%) to obtain the matching with the strongest correlation as the final matching result. It is understood that the threshold parameter is a background program parameter, and generally does not need to be customized by a user to realize a low change frequency.
Then, as described in step S104, a final result is determined to be completed manually through the human-machine interaction interface operation. The user further checks the degree of matching. When a matching degree safety threshold is preset to be 80%, for example, fields needing to be determined manually are filtered out.
And storing the mapping result after manual determination. And if the result needs to be adjusted, the user adjusts the mapping relation through the human-computer interaction interface and then stores the result, and simultaneously sends the adjusted field related mapping relation to the artificial intelligence training computer.
In step S105, the artificial intelligence training computer is responsible for training the neural network. The training computer and the execution machine are distinct discrete machines, wherein the execution machine is a production machine and the training machine is a training machine. The two works are deployed separately and both implement different functions. Here, the result of the first training is sent to the artificial intelligence matching execution machine. At a later stage, optionally, iterative correction training is performed by the mapped samples continually subjected to human intervention, thereby expanding the training set. And then, sending the trained new version neural network to an execution machine to perform mapping relation matching operation. Alternatively, trained computers are typically implemented using GPUs that support hardware acceleration.
As shown in fig. 3, a mapping matching apparatus 300 according to an embodiment of the present invention includes, but is not limited to: the system comprises an input module 301, an artificial intelligence matching module 302, a storage module 303, an artificial intelligence training module 304 and a display module 305.
Similar to the operation flow of the foregoing embodiment of the present invention, the input module 301 may be implemented by a human-machine interface for receiving an input of a user. The input of the source field and the target field 102 is realized by an input module. Optionally, the human-computer interface may display user input.
The artificial intelligence matching module 302 may be implemented by or housed in a computer processor. Through the matching module 302, a main program module or other functional sub-modules for executing artificial intelligence matching operation are provided. The source and destination fields entered by the input module 301 are further entered into the matching module 302 by receiving user input. In the matching module 302, the degree of recognition, i.e., the degree of matching, of the target with the source field is calculated by artificial intelligence.
Further, the result with the highest acquaintance may be output and recorded to the mapping result table, and the result may be stored in the storage module 303. The mapping result table is fed back to the user through the display module 305.
The mapping result contains three parts, namely a source field, a target field and a matching degree. The matching degree is calculated by the neural network, and the final matching result is the matching with the strongest correlation.
The fields that need to be manually determined are filtered out by a preset match security threshold, such as 80%. And the user stores the correct mapping result into the storage module by judging. If the result needs to be adjusted, the user adjusts the mapping relation through the input module, and then the adjusted mapping result is stored in the storage module. Meanwhile, the user sends the adjusted mapping relationship to the artificial intelligence training module 304.
The artificial intelligence training module 304 is a separate module from the artificial intelligence matching module 301. The artificial intelligence training calculations 304 may be implemented by or housed in a computer processor, typically accomplished with a GPU that supports hardware acceleration.
The neural network is trained by the artificial intelligence training module 304. The results of the first training may be sent to artificial intelligence matching module 302. At a later stage, for example in an iterative manner, the manually-intervened mapping sample may be continuously subjected to repeated correction training to expand the training set. Then, the trained new version neural network is sent to the artificial intelligence matching module 302 for mapping matching operation.
The corrected matching result is obtained through artificial intervention, and the corrected matching result is input into the artificial intelligence training module 302 for training as a training material. Thereby, optimization of the matching can be achieved.
Referring now to FIG. 4, a block diagram of a computer system 400 suitable for use in implementing a terminal device of another embodiment of the present invention is shown. The terminal device shown in fig. 4 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 4, the computer system 400 includes a Central Processing Unit (CPU)401 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data necessary for the operation of the system 400 are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
The following components are connected to the I/O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN card, a modem, or the like. The communication section 409 performs communication processing via a network such as the internet. A driver 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 410 as necessary, so that a computer program read out therefrom is mounted into the storage section 408 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 409, and/or installed from the removable medium 411. The computer program performs the above-described functions defined in the system of the present invention when executed by a Central Processing Unit (CPU) 401.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: one or more processors; a storage device to store one or more programs. When executed by the one or more processors, cause the one or more processors to implement the methods or processes in accordance with the above-described embodiments of the invention. Note that the names of these modules do not in some cases constitute a limitation on the modules themselves.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs, and when the one or more programs are executed by a device, the device implements the method or the process for matching the mapping relationship according to the embodiment of the present invention.
According to the invention, the mapping relation matching is carried out by using the artificial intelligent neural network technology, so that the operation of the correlation degree of the incidence relation of the two fields can be realized by training the neural network, and the function which cannot be completed by the traditional simple word spelling matching is realized. Moreover, the function of precipitating experience of an expert system can be further realized by recording the mapping relation which is considered by the user to be incorrect in matching and using the information as a sample for training a computer to train the neural network again.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (19)
1. An artificial intelligence based mapping relationship matching method, comprising:
inputting, via an input device, a source field and a target field into a neural network;
calculating the matching degree of the target field and the source field through the neural network to obtain a mapping result table containing the matching degree;
adjusting the mapping relation in the mapping result table by a user to obtain a mapping result table containing the adjusted mapping relation; and
and training the neural network according to the adjusted mapping relation so as to calculate the matching degree through the trained neural network.
2. The mapping relationship matching method according to claim 1,
in the adjustment of the mapping relation, the matching degree is compared with a preset safety threshold value, so that the mapping relation needing to be adjusted is determined.
3. The mapping relationship matching method according to claim 1,
in the adjustment of the mapping relationship, the adjusted mapping relationship is stored.
4. The mapping relationship matching method according to claim 1,
the training is performed iteratively in an iterative manner.
5. The mapping relationship matching method according to claim 1,
and calculating the matching degree according to the semantics of the source field and the target field.
6. The mapping relationship matching method according to claim 2,
the safety threshold is 80% or greater.
7. The mapping relationship matching method according to claim 1,
inputting the source field and the target field via a human-machine interface as the input device.
8. The mapping relationship matching method according to claim 1,
the training is performed by an artificial intelligence training machine.
9. The mapping relationship matching method according to claim 1,
and storing the mapping relation to be adjusted as a sample for training, and training the sample.
10. The mapping relationship matching method according to claim 1,
the neural network implements computations by the CPU or GPU.
11. The mapping relationship matching method according to claim 1,
the training of the neural network is by training a computer to train an intelligent matching model through artificial intelligence, an
Receiving, by an artificial intelligence execution engine separate from the artificial intelligence training engine, a new trained matching model, and calculating the degree of matching using the trained matching model.
12. A mapping relationship matching apparatus, comprising: an input module, an artificial intelligence matching module, a storage module, an artificial intelligence training module and a display module, wherein,
the input module is used for receiving a source field and a target field;
the artificial intelligence matching module comprises a neural network, a source field and a target field, and is used for receiving the source field and the target field, and calculating the matching degree of the target field and the source field to obtain a mapping result table containing the matching degree;
the storage module is used for storing a mapping result table which is obtained by adjusting the mapping relation in the mapping result table through a user and contains the adjusted mapping relation;
the display module is used for displaying the mapping result table of the adjusted mapping relation; and
the artificial intelligence training module is used for training the neural network according to the adjusted mapping relation, so that the matching degree is calculated through the trained neural network.
13. The mapping relation matching apparatus according to claim 12,
the artificial intelligence training module is further configured for: in the adjustment of the mapping relation, the matching degree is compared with a preset safety threshold value, so that the mapping relation needing to be adjusted is determined.
14. The mapping relation matching apparatus according to claim 12,
the artificial intelligence training module is further configured to iteratively perform the process of training in an iterative manner.
15. The mapping relation matching apparatus according to claim 12,
the neural network calculates the matching degree according to the semantics of the source field and the target field.
16. The mapping relation matching apparatus according to claim 12,
the safety threshold is 80% or greater.
17. The mapping relation matching apparatus according to claim 12,
the storage module is further configured to: and storing the mapping relation to be adjusted as a sample for training, and training the sample.
18. An electronic device for data desensitization, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-11.
19. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011017685.2A CN112199372A (en) | 2020-09-24 | 2020-09-24 | Mapping relation matching method and device and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011017685.2A CN112199372A (en) | 2020-09-24 | 2020-09-24 | Mapping relation matching method and device and computer readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112199372A true CN112199372A (en) | 2021-01-08 |
Family
ID=74006650
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011017685.2A Pending CN112199372A (en) | 2020-09-24 | 2020-09-24 | Mapping relation matching method and device and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112199372A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113568914A (en) * | 2021-07-29 | 2021-10-29 | 上海浦东发展银行股份有限公司 | Data processing method, device, equipment and storage medium |
| CN113722345A (en) * | 2021-09-14 | 2021-11-30 | 上海明略人工智能(集团)有限公司 | Method and device for displaying mapping relation, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190286620A1 (en) * | 2018-03-19 | 2019-09-19 | Perkinelmer Informatics, Inc. | Methods and Systems for Automating Clinical Data Mapping and Transformation |
| CN110427991A (en) * | 2019-07-22 | 2019-11-08 | 联动优势科技有限公司 | A kind of character string matching method and device |
| CN110442568A (en) * | 2019-07-30 | 2019-11-12 | 北京明略软件系统有限公司 | Acquisition methods and device, storage medium, the electronic device of field label |
| CN110955781A (en) * | 2019-11-14 | 2020-04-03 | 北京明略软件系统有限公司 | Model training method and device, and method and device for realizing benchmarking |
| CN111061833A (en) * | 2019-12-10 | 2020-04-24 | 北京明略软件系统有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN111680506A (en) * | 2020-04-28 | 2020-09-18 | 北京三快在线科技有限公司 | External key mapping method and device of database table, electronic equipment and storage medium |
-
2020
- 2020-09-24 CN CN202011017685.2A patent/CN112199372A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190286620A1 (en) * | 2018-03-19 | 2019-09-19 | Perkinelmer Informatics, Inc. | Methods and Systems for Automating Clinical Data Mapping and Transformation |
| CN110427991A (en) * | 2019-07-22 | 2019-11-08 | 联动优势科技有限公司 | A kind of character string matching method and device |
| CN110442568A (en) * | 2019-07-30 | 2019-11-12 | 北京明略软件系统有限公司 | Acquisition methods and device, storage medium, the electronic device of field label |
| CN110955781A (en) * | 2019-11-14 | 2020-04-03 | 北京明略软件系统有限公司 | Model training method and device, and method and device for realizing benchmarking |
| CN111061833A (en) * | 2019-12-10 | 2020-04-24 | 北京明略软件系统有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN111680506A (en) * | 2020-04-28 | 2020-09-18 | 北京三快在线科技有限公司 | External key mapping method and device of database table, electronic equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113568914A (en) * | 2021-07-29 | 2021-10-29 | 上海浦东发展银行股份有限公司 | Data processing method, device, equipment and storage medium |
| CN113722345A (en) * | 2021-09-14 | 2021-11-30 | 上海明略人工智能(集团)有限公司 | Method and device for displaying mapping relation, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10698932B2 (en) | Method and apparatus for parsing query based on artificial intelligence, and storage medium | |
| CN106776936B (en) | Intelligent interaction method and system | |
| US11048762B2 (en) | User-defined automated document feature modeling, extraction and optimization | |
| CN107656996B (en) | Man-machine interaction method and device based on artificial intelligence | |
| CN111159220B (en) | Method and apparatus for outputting structured query statement | |
| CN109325040B (en) | FAQ question-answer library generalization method, device and equipment | |
| US10706030B2 (en) | Utilizing artificial intelligence to integrate data from multiple diverse sources into a data structure | |
| CN110502227A (en) | The method and device of code completion, storage medium, electronic equipment | |
| CN111858913A (en) | Method and system for automatically generating text abstract | |
| EP4086893B1 (en) | Natural language understanding method and device, vehicle and medium | |
| CN107291692B (en) | Artificial intelligence-based word segmentation model customization method, device, equipment and medium | |
| CN117389541B (en) | Configuration system and device for generating template based on dialogue retrieval | |
| CN112199372A (en) | Mapping relation matching method and device and computer readable medium | |
| CN112907358A (en) | Loan user credit scoring method, loan user credit scoring device, computer equipment and storage medium | |
| CN111666379B (en) | Event element extraction method and device | |
| CN114548192A (en) | Sample data processing method and device, electronic equipment and medium | |
| CN113741864B (en) | Automatic semantic service interface design method and system based on natural language processing | |
| CN112582073B (en) | Medical information acquisition method, device, electronic equipment and medium | |
| CN111325031A (en) | Resume parsing method and device | |
| CN114357195A (en) | Knowledge graph-based question-answer pair generation method, device, equipment and medium | |
| KR102019752B1 (en) | Method of providing user interface/ user experience strategy executable by computer and apparatus providing the same | |
| CN117112727A (en) | Large language model fine tuning instruction set construction method suitable for cloud computing service | |
| CN113535125A (en) | Financial demand item generation method and device | |
| CN115795046A (en) | Data processing method, device, system, electronic device and storage medium | |
| CN112966087B (en) | Intelligent question-answering system and method for inspiration materials |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |