CN109947739A - Data power supply management method and device - Google Patents
Data power supply management method and device Download PDFInfo
- Publication number
- CN109947739A CN109947739A CN201810550173.9A CN201810550173A CN109947739A CN 109947739 A CN109947739 A CN 109947739A CN 201810550173 A CN201810550173 A CN 201810550173A CN 109947739 A CN109947739 A CN 109947739A
- Authority
- CN
- China
- Prior art keywords
- meta
- model
- attribute
- file
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This disclosure relates to data power supply management method and device, the description file including ergodic data source；Description file for any data source traversed: the description file is parsed, the mark of the meta-model of the data source is obtained；If meta-model manages the mark for not including the meta-model in file, the description file is parsed, the information for the attribute that the meta-model includes is obtained；The incidence relation of the information for identifying the attribute for including with the meta-model of the meta-model is recorded in meta-model management file.The increase that meta-model is realized by the description file based on data source, can be improved the scalability of meta-model type according to the data power supply management method and device of the embodiment of the present disclosure.
Description
Technical field
This disclosure relates to big data technical field more particularly to a kind of data power supply management method and device.
Background technique
Data source is the data acquisition system for some position being present on local or network.In big data platform, data
The huge number in source is broadly divided into database, file system and message-oriented middleware etc..Data management system can be to entire
Data source in platform is managed collectively, and the wasting of resources is reduced, and facilitates unified Data Audit, data traceability and data prison
Control.
Certain attributes of data source are the attribute for connecting the data source and needing, such as IP address, port numbers, user name and close
Code etc..Certain attributes of data source can optimize data source connection, for example whether can use thread pool, disconnect the reconnection time, connect
The character set etc. connect.The attribute that connection data source needs is known as essential attribute, and the attribute that can optimize data source connection is known as expanding
Open up attribute.The essential attribute of data source and the meta-model in extended attribute composition data source.According to base each in the meta-model of data source
The attribute value of this attribute, can connect data source, according to the attribute value of extended attribute each in the meta-model of data source, can optimize
Data source connection.
In the related technology, it can use relevant database management data source information, by base each in the meta-model of data source
This attribute is remembered as the attribute field in relational data table using the attribute value of each essential attribute of each data source as one
Record is stored in the tables of data.When managing data source information using aforesaid way, if adding a kind of new data source, the data source
The meta-model essential attribute that includes and existing the tables of data attribute field that includes it is inconsistent, then need in the database according to
The essential attribute that the meta-model includes increases a tables of data, to store each essential attribute of the data source in the tables of data
Attribute value；An essential attribute is added or deleted if existing meta-model, then needs to modify the corresponding table structure of the meta-model,
Management business is caused to change larger, scalability is poor.
Summary of the invention
In view of this, the present disclosure proposes a kind of data power supply management method and device, can be improved meta-model type can
Scalability.
According to the one side of the disclosure, a kind of data power supply management method is provided, comprising: the description text in ergodic data source
Part；Description file for any data source traversed: the description file is parsed, the meta-model of the data source is obtained
Mark；If meta-model manages the mark for not including the meta-model in file, the description file is parsed, obtains first mould
The information for the attribute that type includes；The mark of the meta-model is recorded in meta-model management file and the meta-model includes
Attribute information incidence relation.
According to another aspect of the present disclosure, a kind of data source managing device is provided, comprising: file traversal module is used for
The description file in ergodic data source；Identifier acquisition module obtains the meta-model of the data source for parsing the description file
Mark；Data obtaining module is retouched described in parsing if the mark for not including the meta-model in meta-model management file
File is stated, the information for the attribute that the meta-model includes is obtained；Relation record module, in meta-model management file
The incidence relation of the information for the attribute that the mark and the meta-model for recording the meta-model include.
According to another aspect of the present disclosure, a kind of data source managing device is provided, comprising: processor；At storage
Manage the machine readable storage medium of device executable instruction；Wherein, the processor is configured to executing the above method.
According to another aspect of the present disclosure, a kind of machine readable storage medium is provided, contains the executable finger of machine thereon
It enables, for the machine-executable instruction when being called and being executed by processor, the machine-executable instruction promotes the processor
Realize the above method.
By the description file in ergodic data source, for the description file in any data source traversed: being retouched described in parsing
File is stated, the mark of the meta-model of the data source is obtained；If it does not include the mark that meta-model, which manages in file, institute is parsed
Description file is stated, the information for the attribute that the meta-model includes is obtained；First mould is recorded in meta-model management file
The incidence relation of the information for the attribute that the mark of type and the meta-model include, according to the data source of various aspects of the present disclosure embodiment
Management method and device can realize the increase of meta-model based on the description file of data source, compared to passing through increasing in the related technology
Add a tables of data to increase a meta-model newly, do not need modification management business, scalability is preferable.
According to below with reference to the accompanying drawings to detailed description of illustrative embodiments, the other feature and aspect of the disclosure will become
It is clear.
Detailed description of the invention
Comprising in the description and constituting the attached drawing of part of specification and specification together illustrates the disclosure
Exemplary embodiment, feature and aspect, and for explaining the principles of this disclosure.
Fig. 1 shows the flow chart of the data power supply management method according to one embodiment of the disclosure.
Fig. 2 shows the flow charts according to the data power supply management method of one embodiment of the disclosure.
Fig. 3 shows the block diagram according to one embodiment data source managing device of the disclosure.
Fig. 4 shows the block diagram of the data source managing device according to one embodiment of the disclosure.
Fig. 5 is a kind of block diagram for data source managing device shown according to an exemplary embodiment.
Specific embodiment
Various exemplary embodiments, feature and the aspect of the disclosure are described in detail below with reference to attached drawing.It is identical in attached drawing
Appended drawing reference indicate element functionally identical or similar.Although the various aspects of embodiment are shown in the attached drawings, remove
It non-specifically points out, it is not necessary to attached drawing drawn to scale.
Dedicated word " exemplary " means " being used as example, embodiment or illustrative " herein.Here as " exemplary "
Illustrated any embodiment should not necessarily be construed as preferred or advantageous over other embodiments.
In addition, giving numerous details in specific embodiment below to better illustrate the disclosure.
It will be appreciated by those skilled in the art that without certain details, the disclosure equally be can be implemented.In some instances, for
Method, means, element and circuit well known to those skilled in the art are not described in detail, in order to highlight the purport of the disclosure.
Fig. 1 shows the flow chart of the data power supply management method according to one embodiment of the disclosure.The data source that the disclosure provides
Management method can be applied in data management system, as shown in Figure 1, the data power supply management method includes:
S11, the description file in ergodic data source.
The meta-model of data source includes multiple attributes, these attributes can be divided into essential attribute and extended attribute.Based on member
The connection with data source may be implemented in the attribute value of each essential attribute of model, and the attribute value based on each extended attribute of meta-model can be with
Realize the optimization connected to data source.
Description file can be used for describing the meta-model of data source, pass through the available data source that arrives of parsing description file
The information for the attribute that meta-model includes also has determined that out that this describes the meta-model of the corresponding data source of file.
Description file can be the file that is easy to read and write and be easy to machine parsing and generate.Such as XML
(eXtensible Markup Language, extensible markup language) file or JSON (JavaScript Object
Notation, JS object numbered musical notation) file etc., with no restrictions to this disclosure.It should be noted that the analyzing efficiency of JSON file
Higher, XML file more easily describes complicated meta-model.It should be noted that by the number of a data management system management
Description file according to source can be same type of file, and the type of Unify legislation file can be in order to describing the management of file
And parsing.
In the embodiments of the present disclosure, the description file of data source can store in designated position.Designated position, which can be, appoints
What accessible position of data management system.The disclosure for designated position with no restrictions.
In one possible implementation, the description file includes data source label, essential attribute label and extension
Attribute tags, wherein the type information of the data source label corresponding meta-model in source for storing data, the essential attribute
Label is used to store the information of essential attribute, and the extended attribute label is used to store the information of extended attribute.
The type information of meta-model can be used for identifying the type of meta-model.The type of meta-model is different, shows meta-model
Including attribute it is different.In one possible implementation, the type information of meta-model can according to the type of data source and
Version number determines.Such as the type of the corresponding meta-model of data source that txt type, version number are 1.0 can be Class1, txt class
The type of the corresponding meta-model of the data source that type, version number are 2.0 can be type 2, message-oriented middleware kafka type, version
It number can be type 3 for the type of the 1.0 corresponding meta-model of data source, MySQL type, the data source pair that version number is 1.0
The type for the meta-model answered can be type 4 etc..File is described by parsing, meta-model can be obtained from data source label
Type information.
In one possible implementation, the essential attribute label includes the information of one or more essential attributes,
The information of each essential attribute includes the mark and constraint condition of the essential attribute.File, Ke Yicong are described by parsing
The information of each essential attribute is obtained in essential attribute label.
In one possible implementation, the extended attribute label includes the information of one or more extended attributes,
The information of each extended attribute includes the mark and constraint condition of the extended attribute.File is described by parsing, it can be from extension
The information of each extended attribute is obtained in attribute tags.
Extended attribute can optimize the connection of data source, may determine whether that thread pool, number can be used according to extended attribute
According to the disconnection reconnection time in source, the character set etc. of data source connection.Business scenario is different, the extended attribute that data source connection needs
May also be different, it there may come a time when not needing extended attribute.In one possible implementation, extended attribute in file is described
Information can be null value, indicate no extended attribute.
In the related technology when managing the extended attribute of data source, need each self-generating of the extended attribute of every kind of data source
One tables of data.There are problems that two, first: needing to increase multiple tables of data (being approximately equal to data source category), increase data
The difficulty of library entity management；Second: leading to " Sparse ".
According to the data power supply management method of the embodiment of the present disclosure: on the one hand, the extended attribute in description file can be passed through
Increase in label, the information of modification and deletion extended attribute, realizes addition, modification and the deletion of extended attribute in meta-model, energy
Enough flexible management extended attributes.On the other hand, after describing file generated, the most of the time describes file and does not have any change
It is dynamic, it does not need to be safeguarded using database, reduces the complexity between each entity.Meanwhile when some data source needs
Some extended attribute, can only store the extended attribute mark and corresponding attribute value, for the extension category not used
Property does not have to then store, the problem of so as to avoid " Sparse ".
To improve safety, each attribute is provided with corresponding constraint condition, such as port numbers can only be number, IP address
Have to comply with certain format.In one example, constraint condition include: whether for encrypted fields, whether be required field, whether
For read only field, the set for whether needing specified default value, whether can only filling in certain values, the canonical constraint of attribute value, attribute
One or more of the maximum length of value constrains and the minimum length of attribute value constrains etc..
Wherein, a value, i.e. default value can be automatically provided before the attribute value for getting attribute, allow data source at this
There is content under attribute, facilitates management.For example, for this attribute of the disconnection reconnection time of data source, in not set corresponding category
Property value when, data source disconnect after may constantly try to connection data source event, after for the attribute default settings, that
When turn-off time reaches default value, connection data source event can be just generated, resource can be so saved.
It in one possible implementation, further include extension functional label, the extension function in the description file
For label for storing the extension functional information, the extension functional information is including extension Function Identification and realizes the extension function
The class of energy or the essential information of function.Wherein, extension based on meta-models function includes data source continuity testing, data preview, company
Connect url, SQL execution etc..File is described by parsing, extension functional information can be obtained from extension functional label.
S12 parses the description file, obtains the mark of the meta-model of the data source.
The mark of meta-model can identify unique meta-model.Can be by the description file of parsing data source, obtaining should
The mark of the meta-model of data source.In one possible implementation, the mark of meta-model can be the title of description file
Or md5 value etc., with no restrictions to this disclosure.By meta-model be identified as description file title for, available time
This, is referred to as the mark of meta-model by the title for the description file gone through.With the md5 value for being identified as description file of meta-model
For, md5 conversion can be carried out to the content of the description file traversed, obtain the md5 value of description file, which is made
For the mark of meta-model.
S13 parses the description file, obtains if not including the mark of the meta-model in meta-model management file
The information for the attribute that the meta-model includes.
Meta-model manages information of the file for the data source of record management, such as: it can remember in meta-model management file
Incidence relation between the information for the attribute that the mark and the meta-model for recording the meta-model of data source include.Meta-model manages file
In include multiple incidence relations, each incidence relation correspond to a meta-model.It can manage file by meta-model and find and work as
The meta-model of all data sources of preceding moment management.
In one possible implementation, the mark and member of meta-model are stored in meta-model management file by key-value pair
The incidence relation of the information for the attribute that model includes.The key of the key-value pair is the mark of the meta-model, the key-value pair
Value is the information for the attribute that the meta-model includes, wherein the information of the attribute includes the mark and constraint condition of attribute.Key
Value has the characteristics that inquiry velocity is fast, store data volume is big to the distributed storage of form, supports high concurrent, it is easy to accomplish high speed
Inquiry.
When not including the mark of acquired meta-model in meta-model management file, need to increase the meta-model to manage
Corresponding data source.File can be described by parsing this, get the information for the attribute that meta-model includes.It can be according to these
The information of attribute determines meta-model, to manage corresponding data source.
In a kind of possible implementation, the information of attribute includes the mark of attribute.According to the mark of attribute, can determine
Which attribute meta-model includes, that is, determines the concrete composition of a meta-model.
In one possible implementation, the information of the attribute recorded in metadata management file further includes the pact of attribute
Beam condition.Attribute value can be verified according to the constraint condition of attribute, improve safety.
In the related technology, it needs to add trigger to each attribute of meta-model in tables of data, utilizes the touching of database
Device is sent out to judge whether the attribute value of each attribute meets corresponding constraint condition, the writing speed of database is reduced, increases
The pressure of database.In the embodiment of the present disclosure, the mark for the attribute that meta-model includes can be determined according to metadata management file
And constraint condition, attribute value can be obtained according to the mark of attribute, it can be without about when database is written in the attribute value
The verification of beam condition.Constraint condition can be verified in client, can also be placed in business function and be verified, and will be verified
Pressure dissipation into each client and business function, improve the writing speed of database, increase handling up for database
Amount.
The information for the attribute that meta-model includes includes the information of essential attribute and the information of extended attribute.The letter of essential attribute
Breath includes the mark and constraint condition of essential attribute, and the information of extended attribute includes the mark and constraint condition of extended attribute.Ginseng
According to step S11 it is found that by parsing description file, one or more bases can be obtained from the essential attribute label of description file
The information of this attribute obtains the information of one or more extended attributes from the extended attribute label of description file.
In one possible implementation, if including the mark in meta-model management file, this is not handled
File is described, such as ignores or deletes this and describe file.
S14 records the attribute that the mark of the meta-model includes with the meta-model in meta-model management file
Information incidence relation.
The attribute that the meta-model obtained in the mark and S13 obtained in record S12 in meta-model management file includes
The administrative relationships of information, to be managed to the data source.
After the completion of the description file traversal of data source, be described file is had recorded in meta-model management file and is corresponded to
Data source information.
Based on example disclosed above, when there is a new data source to be managed, it is only necessary to increase retouching for the data source
File is stated, compared to by way of managing newly-increased data source, not needing to repair increasing a tables of data in the related technology
Change management business, scalability is preferable.
It should be noted that being performed both by step S12 extremely for the description file in any data source traversed in step S11
Step S14, until having traversed the description file of all data sources.
In one possible implementation, the description file in ergodic data source can be carried out according to certain time interval.When
When adding a kind of new data source, it is only necessary to which the description file of the data source, which is stored in designated position, can be realized meta-model
Addition, does not need the service of restarting, can come into force immediately.Above-mentioned time interval is smaller, instantaneity is better.
Fig. 2 shows the flow charts according to the data power supply management method of one embodiment of the disclosure.As shown in Fig. 2, for traversal
The description file of the data source arrived, after getting the mark of meta-model of the data source, this method further include:
S15, by the identification record of the meta-model in this corresponding mark file of traversal.
S16 traverses the meta-model in the meta-model management file after having traversed the description file of the data source
Mark.
S17, for the mark of any meta-model traversed, if the mark of the meta-model is not deposited in the mark file
The attribute that the mark of the meta-model includes with the associated meta-model is then being deleted from meta-model management file
Information.
It is illustrated by taking the title for being identified as description file of meta-model as an example.Assuming that under initial situation, in designated position
Store the description file 3 of the description file 1 of data source 1, the description file 2 and data source 3 of data source 2.First mould of data source 1
The information for the attribute that the title f1 for being identified as description file 1 and the meta-model of type include is m1, the mark of the meta-model of data source 2
The information for the attribute that knowledge is the title f2 of description file 2 and the meta-model includes is m2, and being identified as the meta-model of data source 3 is retouched
It states the title f3 of file 3 and the information of attribute that the meta-model includes is m3, have recorded f1's and m1 in meta-model management file
The incidence relation of incidence relation, the incidence relation of f2 and m2 and f3 and m3.
The description file 3 of file 4, deletion data source 3 is described in designated position addition data source 4, and modifies data source 2
Description file 2 in the content title of file 2 (description constant) in the case where:
Traversal description file 1, description file 2 and description file 4.When traversing description file 1, the member of data source 1 is obtained
The mark f1 of model；F1 is recorded in this to traverse in corresponding mark file.Due to including f1 in meta-model management file, because
This ignores this and describes file.When traversing description file 2, described with reference to file 1, f2 is recorded in this and traverses corresponding mark
In file.When traversing description file 4, the mark f4 of the meta-model of data source 4 is obtained；It is corresponding that f4 is recorded in this traversal
It identifies in file.Due to not including f4 in meta-model management file, the meta-model that parsing description file 4 obtains data source 4 includes
Attribute information m4；The corresponding relationship of f4 and m4 is recorded in meta-model management file.At this point, this traverses corresponding mark
File includes f1, f2 and f4.The incidence relation of f1 and m1, the incidence relation of f2 and m2, f3 are had recorded in meta-model management file
With the incidence relation and f4 of m3 and the incidence relation of m4.
Traverse the mark of the meta-model in the meta-model management file: f1, f2, f3 and f4.F1, f2 and f4 are in mark text
Exist in part, f3 is not present in mark file, and the incidence relation of f3 and m3 is deleted from meta-model management file.This
When, meta-model manages the association that incidence relation, f2 and m2 that f1 and m1 is had recorded in meta-model management file are had recorded in file
The incidence relation of relationship and f4 and m4.
It can be seen that with meta-model when being identified as the title for describing file, in the description of designated position addition data source 4
File 4, delete data source 3 description file 3, and modify data source 2 description file 2 in content in the case where, meta-model
The meta-model that data source 4 is increased in management file, deletes the meta-model of data source 3, the meta-model packet in unmodified data source 2
The information of the attribute included had both realized the increase and deletion of meta-model, in turn avoided directly changing the attribute that meta-model includes
Information.Therefore, in the information for not allowing directly to change the attribute that meta-model includes (for example, one extension of addition or deletion belongs to
Property) scene in, can using description file name be referred to as identify.
It is illustrated by taking the md5 value for being identified as description file of meta-model as an example.Md5 algorithm can be the word of any length
Symbol string is transformed into the hexadecimal number word string of certain length (such as 16,32).If original character string has done any change,
The numeric string obtained through md5 algorithm operation can change.The md5 value of description file is that description file is converted through md5 algorithm
The value obtained afterwards, describing any modification in file can all cause the md5 value for describing file to change.
Assuming that storing the description file 2 of the description file 1 of data source 1, data source 2 under initial situation, in designated position
With the description file 3 of data source 3, the md5 value v1 for being identified as description file 1 and the meta-model of the meta-model of data source 1 include
The information of attribute be m1, the md5 value v2 and the meta-model category that includes for being identified as description file 2 of the meta-model of data source 2
Property information be m2, the md5 value v3 and the meta-model attribute that includes for being identified as description file 3 of the meta-model of data source 3
Information is m3, has recorded the incidence relation of v1 and m1, the incidence relation of v2 and m2 and v3 and m3 in meta-model management file
Incidence relation.
The description file 3 of file 4, deletion data source 3 is described in designated position addition data source 4, and modifies data source 2
Description file 2 in the content md5 value of file 2 (description v6 is become from v2) in the case where:
Traversal description file 1, description file 2 and description file 4.When traversing description file 1, the member of data source 1 is obtained
V1 is recorded in this and traversed in corresponding mark file by the mark v1 of model, due to including v1 in meta-model management file, because
This ignores this and describes file.When traversing description file 2, the mark v6 of the meta-model of data source 2 is obtained, v6 is recorded in this
It traverses in corresponding mark file, due to not including v6 in meta-model management file, parsing description file 2 obtains data source 2
The information m6 for the attribute that meta-model includes records the corresponding relationship of v6 and m6 in meta-model management file.Traverse description text
When part 4, the mark v4 of the meta-model of data source 4 is obtained, v4 is recorded in this and is traversed in corresponding mark file, due to first mould
It does not include v4 that type, which manages in file, the information m4 for the attribute that the meta-model of the parsing description acquisition data source 4 of file 4 includes, in member
The corresponding relationship of v4 and m4 is recorded in model management file.At this point, this traverse in corresponding mark file including v1, v6 and
v4.Meta-model management file in have recorded the incidence relation of v1 and m1, the incidence relation of v2 and m2, v3 and m3 incidence relation,
The incidence relation and v6 of v4 and m4 and the incidence relation of m6.
Traverse the mark of the meta-model in the meta-model management file: v1, v2, v3, v4 and v6.V1, v4 and v6 are being marked
Know in file and exist, v2 and v3 are not present in mark file, and the association of v2 and m2 is deleted from meta-model management file
The incidence relation of relationship and v3 and m3.At this point, have recorded the incidence relation of v1 and m1 in meta-model management file, v4 and m4's
The incidence relation of incidence relation and v6 and m6.
It can be seen that when the md5 value for being identified as description file of meta-model, in the description of designated position addition data source 4
File 4, delete data source 3 description file 3, and modify data source 2 description file 2 in content in the case where, meta-model
The meta-model that data source 4 is increased in management file, deletes the meta-model of data source 3, has updated the meta-model packet of data source 2
The information of the attribute included realizes the increase, deletion and update of meta-model.Therefore, include in permission directly change meta-model
It, can be using the md5 value conduct of description file in the scene of the information (for example, one extended attribute of addition or deletion) of attribute
Mark.
Fig. 3 shows the block diagram according to one embodiment data source managing device of the disclosure.As shown in figure 3, described device 30 is wrapped
It includes:
File traversal module 31, the description file for ergodic data source；
Identifier acquisition module 32 obtains the mark of the meta-model of the data source for parsing the description file；
Data obtaining module 33, if the mark for not including the meta-model in meta-model management file, parses institute
Description file is stated, the information for the attribute that the meta-model includes is obtained；
Relation record module 34, for recording the mark and the member of the meta-model in meta-model management file
The incidence relation of the information for the attribute that model includes.
In one possible implementation, the description file is the file of XML type or JSON type；The member mould
The information for the attribute that type includes includes the information of essential attribute and the information of extended attribute；The description file includes data source mark
Label, essential attribute label and extended attribute label, wherein the essential attribute label is used to store the letter of the essential attribute
Breath；The extended attribute label is used to store the information of the extended attribute.
In one possible implementation, the essential attribute label includes the information of one or more essential attributes,
The information of each essential attribute includes the mark and constraint condition of the essential attribute；The extended attribute label include one or
The information of multiple extended attributes, the information of each extended attribute include the mark and constraint condition of the extended attribute.
In one possible implementation, the meta-model is stored by key-value pair in the meta-model management file
The incidence relation of the information for the attribute that mark and the meta-model include；The key of the key-value pair is the mark of the meta-model；
The value of the key-value pair is the information for the attribute that the meta-model includes, and the information of the attribute includes the mark and constraint of attribute
Condition.
Fig. 4 shows the block diagram of the data source managing device according to one embodiment of the disclosure.As shown in figure 4, in a kind of possibility
Implementation in, the device 30 further include:
Identification record module 35, for traversing in corresponding mark file the identification record of the meta-model at this.
In one possible implementation, the device 30 further include:
Spider module 36 is identified, for traversing the mark of the meta-model in the meta-model management file；
Information deletion module 37, for the mark for any meta-model traversed, if the mark of the meta-model is in institute
It states and is not present in mark file, then delete the mark and the associated member of the meta-model from meta-model management file
The information for the attribute that model includes.
In one possible implementation, the meta-model be identified as it is described description file md5 value or be described
The title of file is described.
By the description file in ergodic data source, for the description file in any data source traversed: being retouched described in parsing
File is stated, the mark of the meta-model of the data source is obtained；If it does not include the mark that meta-model, which manages in file, institute is parsed
Description file is stated, the information for the attribute that the meta-model includes is obtained；First mould is recorded in meta-model management file
The incidence relation of the information for the attribute that the mark of type and the meta-model include is filled according to the data source control of the embodiment of the present disclosure
The increase that meta-model can be realized based on the description file of data source is set, compared in the related technology by increasing a tables of data
It increases a meta-model newly, does not need modification management business, scalability is preferable.
Fig. 5 is a kind of block diagram for data source managing device 900 shown according to an exemplary embodiment.Reference Fig. 5,
The device 900 may include processor 901, the machine readable storage medium 902 for being stored with machine-executable instruction.Processor 901
It can be communicated via system bus 903 with machine readable storage medium 902.Also, processor 901 passes through read machine readable storage
Machine-executable instruction corresponding with data source control logic is in medium 902 to execute data power supply management method described above.
Machine readable storage medium 902 referred to herein can be any electronics, magnetism, optics or other physical stores
Device may include or store information, such as executable instruction, data, etc..For example, machine readable storage medium may is that
RAM (Radom Access Memory, random access memory), volatile memory, nonvolatile memory, flash memory, storage are driven
Dynamic device (such as hard disk drive), solid state hard disk, any kind of storage dish (such as CD, dvd) or similar storage are situated between
Matter or their combination.
The presently disclosed embodiments is described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport
In the principle, practical application or technological improvement to the technology in market for best explaining each embodiment, or lead this technology
Other those of ordinary skill in domain can understand each embodiment disclosed herein.
Claims (15)
1. a kind of data power supply management method, which is characterized in that the described method includes:
The description file in ergodic data source；
Description file for any data source traversed:
The description file is parsed, the mark of the meta-model of the data source is obtained；
If meta-model manages the mark for not including the meta-model in file, the description file is parsed, obtains first mould
The information for the attribute that type includes；
The information for identifying the attribute for including with the meta-model of the meta-model is recorded in meta-model management file
Incidence relation.
2. the method according to claim 1, wherein
The description file is the file of XML type or JSON type；
The information for the attribute that the meta-model includes includes the information of essential attribute and the information of extended attribute；
The description file includes data source label, essential attribute label and extended attribute label, wherein
The essential attribute label is used to store the information of the essential attribute；
The extended attribute label is used to store the information of the extended attribute.
3. according to the method described in claim 2, it is characterized in that,
The essential attribute label includes the information of one or more essential attributes, and the information of each essential attribute includes the base
The mark and constraint condition of this attribute；
The extended attribute label includes the information of one or more extended attributes, and the information of each extended attribute includes the expansion
Open up the mark and constraint condition of attribute.
4. the method according to claim 1, wherein
The attribute for including with the meta-model by the mark that key-value pair stores the meta-model in the meta-model management file
Information incidence relation；
The key of the key-value pair is the mark of the meta-model；
The value of the key-value pair is the information of the meta-model attribute that includes, the information of the attribute include attribute mark and
Constraint condition.
5., should according to the method described in claim 4, it is characterized in that, after obtaining the mark of the meta-model of the data source
Method further include:
By the identification record of the meta-model in this corresponding mark file of traversal.
6. described according to the method described in claim 5, it is characterized in that, after having traversed the description file of the data source
Method further include:
Traverse the mark of the meta-model in the meta-model management file；
For the mark of any meta-model traversed, if the mark of the meta-model is not present in the mark file, from
The information for identifying the attribute for including with the associated meta-model of the meta-model is deleted in the meta-model management file.
7. according to the method described in claim 6, it is characterized in that, the md5 for being identified as the description file of the meta-model
Value is the title for describing file.
8. a kind of data source managing device, which is characterized in that described device includes:
File traversal module, the description file for ergodic data source；
Identifier acquisition module obtains the mark of the meta-model of the data source for parsing the description file；
Data obtaining module, if the mark for not including the meta-model in meta-model management file, parses the description
File obtains the information for the attribute that the meta-model includes；
Relation record module, for managing the mark for recording the meta-model in file and the meta-model packet in the meta-model
The incidence relation of the information of the attribute included.
9. device according to claim 8, which is characterized in that
The description file is the file of XML type or JSON type；
The information for the attribute that the meta-model includes includes the information of essential attribute and the information of extended attribute；
The description file includes data source label, essential attribute label and extended attribute label, wherein
The essential attribute label is used to store the information of the essential attribute；
The extended attribute label is used to store the information of the extended attribute.
10. device according to claim 9, which is characterized in that
The essential attribute label includes the information of one or more essential attributes, and the information of each essential attribute includes the base
The mark and constraint condition of this attribute；
The extended attribute label includes the information of one or more extended attributes, and the information of each extended attribute includes the expansion
Open up the mark and constraint condition of attribute.
11. device according to claim 8, which is characterized in that
The attribute for including with the meta-model by the mark that key-value pair stores the meta-model in the meta-model management file
Information incidence relation；
The key of the key-value pair is the mark of the meta-model；
The value of the key-value pair is the information of the meta-model attribute that includes, the information of the attribute include attribute mark and
Constraint condition.
12. device according to claim 11, which is characterized in that the device further include:
Identification record module, for traversing in corresponding mark file the identification record of the meta-model at this.
13. device according to claim 12, which is characterized in that described device further include:
Spider module is identified, for traversing the mark of the meta-model in the meta-model management file；
Information deletion module, for the mark for any meta-model traversed, if the mark of the meta-model is in the mark
It is not present in file, then manages the mark for deleting the meta-model in file and the associated meta-model packet from the meta-model
The information of the attribute included.
14. device according to claim 13, which is characterized in that the meta-model is identified as the description file
Md5 value is the title for describing file.
15. a kind of data source managing device characterized by comprising
Processor and machine readable storage medium, the machine readable storage medium are stored with machine-executable instruction, the place
Reason device executes the machine-executable instruction to realize method described in any one of claim 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810550173.9A CN109947739B (en) | 2018-05-31 | 2018-05-31 | Data source management method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810550173.9A CN109947739B (en) | 2018-05-31 | 2018-05-31 | Data source management method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109947739A true CN109947739A (en) | 2019-06-28 |
| CN109947739B CN109947739B (en) | 2021-10-15 |
Family
ID=67006311
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810550173.9A Active CN109947739B (en) | 2018-05-31 | 2018-05-31 | Data source management method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109947739B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112860793A (en) * | 2021-02-03 | 2021-05-28 | 浪潮云信息技术股份公司 | Method for realizing metadata synchronization between different source databases |
| CN115168363A (en) * | 2022-07-29 | 2022-10-11 | 北京远舢智能科技有限公司 | Metadata processing method and device, electronic equipment and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201522706U (en) * | 2009-09-09 | 2010-07-07 | 中国工商银行股份有限公司 | Metadata management system |
| CN102023979A (en) * | 2009-09-09 | 2011-04-20 | 中国工商银行股份有限公司 | Meta-data management method and system |
| CN102567932A (en) * | 2010-12-30 | 2012-07-11 | 北京亿阳信通软件研究院有限公司 | Method, device and system for managing comprehensive resources |
| US20130117277A1 (en) * | 2011-11-09 | 2013-05-09 | Microsoft Corporation | Aggregating distributed sources of application guidance information |
| US20150143503A1 (en) * | 2012-08-31 | 2015-05-21 | Salesforce.Com, Inc. | Systems and methods for content management in an on-demand environment |
| CN105303326A (en) * | 2015-11-13 | 2016-02-03 | 上海交通大学 | Product full-life-cycle information system based on resource meta-model |
| CN105340240A (en) * | 2013-01-29 | 2016-02-17 | 惠普发展公司，有限责任合伙企业 | Methods and systems for shared file storage |
| CN105653691A (en) * | 2015-12-29 | 2016-06-08 | 珠海城建节能科技有限公司 | Information resource management method and management apparatus |
| CN105677875A (en) * | 2016-01-11 | 2016-06-15 | 清华大学 | Visual modeling platform based on unified constraint of normal data elements |
| CN106227898A (en) * | 2016-08-31 | 2016-12-14 | 江苏艾倍科科技股份有限公司 | A kind of Big Dipper moves wearable device data and administers platform and data administering method thereof |
| CN106294776A (en) * | 2016-08-12 | 2017-01-04 | 北京东方车云信息技术有限公司 | A kind of data processing method and device |
| CN106570018A (en) * | 2015-10-10 | 2017-04-19 | 阿里巴巴集团控股有限公司 | Serialization method and apparatus, deserialization method and apparatus, serialization and deserialization system, and electronic device |
| CN107423211A (en) * | 2017-03-15 | 2017-12-01 | 中国互联网络信息中心 | A kind of SDNS interfaces automatization test system and method |
| CN108052618A (en) * | 2017-12-15 | 2018-05-18 | 北京搜狐新媒体信息技术有限公司 | Data managing method and device |
-
2018
- 2018-05-31 CN CN201810550173.9A patent/CN109947739B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201522706U (en) * | 2009-09-09 | 2010-07-07 | 中国工商银行股份有限公司 | Metadata management system |
| CN102023979A (en) * | 2009-09-09 | 2011-04-20 | 中国工商银行股份有限公司 | Meta-data management method and system |
| CN102567932A (en) * | 2010-12-30 | 2012-07-11 | 北京亿阳信通软件研究院有限公司 | Method, device and system for managing comprehensive resources |
| US20130117277A1 (en) * | 2011-11-09 | 2013-05-09 | Microsoft Corporation | Aggregating distributed sources of application guidance information |
| US20150143503A1 (en) * | 2012-08-31 | 2015-05-21 | Salesforce.Com, Inc. | Systems and methods for content management in an on-demand environment |
| CN105340240A (en) * | 2013-01-29 | 2016-02-17 | 惠普发展公司，有限责任合伙企业 | Methods and systems for shared file storage |
| CN106570018A (en) * | 2015-10-10 | 2017-04-19 | 阿里巴巴集团控股有限公司 | Serialization method and apparatus, deserialization method and apparatus, serialization and deserialization system, and electronic device |
| CN105303326A (en) * | 2015-11-13 | 2016-02-03 | 上海交通大学 | Product full-life-cycle information system based on resource meta-model |
| CN105653691A (en) * | 2015-12-29 | 2016-06-08 | 珠海城建节能科技有限公司 | Information resource management method and management apparatus |
| CN105677875A (en) * | 2016-01-11 | 2016-06-15 | 清华大学 | Visual modeling platform based on unified constraint of normal data elements |
| CN106294776A (en) * | 2016-08-12 | 2017-01-04 | 北京东方车云信息技术有限公司 | A kind of data processing method and device |
| CN106227898A (en) * | 2016-08-31 | 2016-12-14 | 江苏艾倍科科技股份有限公司 | A kind of Big Dipper moves wearable device data and administers platform and data administering method thereof |
| CN107423211A (en) * | 2017-03-15 | 2017-12-01 | 中国互联网络信息中心 | A kind of SDNS interfaces automatization test system and method |
| CN108052618A (en) * | 2017-12-15 | 2018-05-18 | 北京搜狐新媒体信息技术有限公司 | Data managing method and device |
Non-Patent Citations (1)
| Title |
|---|
| 杨阳: "一种异构数据描述与转换框架的研究与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112860793A (en) * | 2021-02-03 | 2021-05-28 | 浪潮云信息技术股份公司 | Method for realizing metadata synchronization between different source databases |
| CN115168363A (en) * | 2022-07-29 | 2022-10-11 | 北京远舢智能科技有限公司 | Metadata processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109947739B (en) | 2021-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9165031B2 (en) | Retrieving stored data using a web service | |
| US8312128B2 (en) | Identification of management information base object identifiers supported by a managed device | |
| TWI483138B (en) | Method for processing and verifying remote dynamic data, system using the same, and computer-readable medium | |
| JP5600736B2 (en) | Database management method and system | |
| CN107958079A (en) | Aggregate file delet method, system, device and readable storage medium storing program for executing | |
| US20160364407A1 (en) | Method and Device for Responding to Request, and Distributed File System | |
| CN109522283B (en) | Method and system for deleting repeated data | |
| US20140101713A1 (en) | Data mapping using trust services | |
| WO2018205981A1 (en) | Metadata management | |
| WO2018233630A1 (en) | Fault discovery | |
| WO2023131218A1 (en) | Graph data storage | |
| JP2012174096A (en) | Computer system and data management method | |
| CN101335756A (en) | Systems and methods for snmp access | |
| WO2011029244A1 (en) | Method and device for managing association of network resources | |
| WO2021017422A1 (en) | Index creation method in block chain type account book, device and apparatus | |
| JP2017521778A (en) | Method, computer program, and exception engine for handling data quality exceptions | |
| CN107103011B (en) | Method and device for realizing terminal data search | |
| US10212067B2 (en) | Dynamic symbolic links for referencing in a file system | |
| US9128962B2 (en) | View based table replacement for applications | |
| CN113704790A (en) | Abnormal log information summarizing method and computer equipment | |
| CN101576919B (en) | Mark generating method and device | |
| CN112052222B (en) | Heterogeneous object storage cluster access method, device, equipment and storage medium | |
| CN109947739A (en) | Data power supply management method and device | |
| CN101582881A (en) | Method and device for controlling access | |
| CN109063061B (en) | Cross-distributed system data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |