CN110032575A - Data query method, apparatus, equipment and storage medium - Google Patents
Data query method, apparatus, equipment and storage medium Download PDFInfo
- Publication number
- CN110032575A CN110032575A CN201910298551.3A CN201910298551A CN110032575A CN 110032575 A CN110032575 A CN 110032575A CN 201910298551 A CN201910298551 A CN 201910298551A CN 110032575 A CN110032575 A CN 110032575A
- Authority
- CN
- China
- Prior art keywords
- data
- query
- metadata information
- sql
- data source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention provides a kind of data query method, device, equipment and storage medium, this method comprises: user has sent SQL query statement by user end to server, after server receives the SQL query statement, it is parsed to obtain corresponding AST first, then the metadata information of the data of inquiry is obtained from AST, query result is got from each different database according to metadata information, it is then returned to client, it realizes while supporting the function of being inquired the multi-data source of data system, the content that database is stored is not limited yet, it facilitates user and carries out data query on multi-data source, set expandability is preferable.
Description
Technical field
The present invention relates to technical field of data administration more particularly to a kind of data query method, apparatus, equipment and storage to be situated between
Matter.
Background technique
Under the data management scene of large enterprise, a variety of different databases can be used according to specific use demand
System, such as: MySQL, Hive, HBase, Oracle, Elasticsearch, MongoDB etc..Developer needs to learn this
The learning cost of a little system usage modes is higher, and inconvenient using configuration.
In the prior art, a kind of automatic routing database inquiry system based on query resource and accuracy is provided, including
Structured query language (Structured Query Language, SQL) standardized query engine modules, unified inquiry enter
Mouth mold block, query result cache module and the data transmission module across structure, SQL standard query engine module includes for mentioning
For the HIVE of basic data, the HBase for providing object acquisition and high speed objects services, for providing data modeling and model
The MySQL of service, the ElasticSearch for providing Cube, for providing the MongoDB of allocation list and Meta information；Institute
State data buffer storage of the query result cache module for unified inquiry inlet module；The SQL standard query engine module
The data transmission module with the unified inquiry inlet module and across structure is connected respectively, by the type to data acquisition into
The segmentation of row engine and appropriateness use, and real data engine is hidden to actual user, provide data service for user.
However, the SQL standard query engine module in above scheme includes for providing the HIVE of basic data, being used for
There is provided the HBase of object acquisition and high speed objects services, the MySQL for providing data modeling and model service, for providing
The ElasticSearch of Cube, for providing the MongoDB of allocation list and Meta information, when the realization of this method, different type
The limitation of the data type of required storage in database, causes data system scalability lower.
Summary of the invention
The present invention provides a kind of data query method, apparatus, equipment and storage medium, different in the prior art for solving
The limitation of the data type of required storage in types of database, the problem for causing data system scalability lower.
First aspect present invention provides a kind of data query method, which comprises
Receive the first SQL query statement that client is sent；
First SQL query statement is parsed, corresponding first AST is obtained；
According to the first AST, the first metadata letter of the first data of the first SQL query statement inquiry is obtained
Breath；
Query result is obtained according to first metadata information, and the query result is returned into the client.
In a kind of concrete implementation mode, the AST includes the first table name belonging to first data, the basis
First AST obtains the first metadata information of the first data of the first SQL query statement inquiry, comprising:
First table name is extracted from the first AST；
According to the corresponding relationship between first table name and table name and metadata information, the first metadata letter is obtained
Breath.
It is described that query result is obtained according to first metadata information in a kind of concrete implementation mode, comprising:
According to first metadata information, the first data source is obtained；
It carries out first SQL query statement being routed to first data source by SQL router；
The query result is obtained from first data source query, the query result includes first data.
Further, the method also includes:
If first data source does not support SQL statement, first SQL query statement is converted, is converted to
The query statement that first data source is supported.
In a kind of concrete implementation mode, the method also includes:
Each data source is scanned according to predetermined period, the corresponding metadata information of each data source being locally stored is carried out
Synchronous, each metadata information includes at least data source types, database name and table name.
In another concrete implementation mode, the method also includes:
The query result is stored, and the query result is identified by first identifier.
It is optionally, described that the query result is returned into the client, comprising:
The query result is packaged into JSON and returns to the client.
Second aspect of the present invention provides a kind of data query device, and described device includes:
Receiving module, for receiving the first structure query language SQL query statement of client transmission；
Processing module obtains corresponding first abstract syntax tree for parsing first SQL query statement
AST；
The processing module is also used to obtain the first number of the first SQL query statement inquiry according to the first AST
According to the first metadata information；
Enquiry module, for obtaining query result according to first metadata information；
Sending module, for the query result to be returned to the client.
In a kind of concrete implementation mode, the AST includes the first table name belonging to first data, the processing
Module is specifically used for:
First table name is extracted from the first AST；
According to the corresponding relationship between first table name and table name and metadata information, the first metadata letter is obtained
Breath.
10, device according to claim 8, which is characterized in that the enquiry module is specifically used for:
According to first metadata information, the first data source is obtained；
It carries out first SQL query statement being routed to first data source by SQL router；
The query result is obtained from first data source query, the query result includes first data.
11, device according to claim 10, which is characterized in that the processing module is also used to:
If first data source does not support SQL statement, first SQL query statement is converted, is converted to
The query statement that first data source is supported.
12, device according to any one of claims 8 to 11, which is characterized in that the processing module is also used to:
Each data source is scanned according to predetermined period, the corresponding metadata information of each data source being locally stored is carried out
Synchronous, each metadata information includes at least data source types, database name and table name.
13, device according to any one of claims 8 to 11, which is characterized in that described device further include:
Memory module carries out the query result for storing to the query result, and by first identifier
Mark.
14, device according to any one of claims 8 to 11, which is characterized in that the sending module is specifically used for:
The query result is packaged into JSON and returns to the client.
Third aspect present invention provides a kind of server, comprising:
Receiver, processor, memory and transmitter；
Memory calls the computer program of memory storage for storing computer program and data, the processor,
To execute the data query method of first aspect any embodiment offer.
Fourth aspect present invention provides a kind of computer readable storage medium, and the computer readable storage medium includes meter
Calculation machine program, the data that the computer program is provided when being executed by processor for executing any implementation of first aspect
Querying method.
A kind of data query method, apparatus provided in an embodiment of the present invention, equipment and storage medium, user pass through client
SQL query statement is had sent to server, after server receives the SQL query statement, is parsed to obtain first pair
Then the AST answered obtains the metadata information of the data of inquiry from AST, according to metadata information from each different data
Query result is got in library, is then returned to client, realizes while supporting to inquire the multi-data source of data system
Function, do not limit the content that database is stored yet, facilitate user and carry out data query on multi-data source, system extension
Property is preferable.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair
Bright some embodiments for those of ordinary skill in the art without any creative labor, can be with
It obtains other drawings based on these drawings.
Fig. 1 is the flow diagram of data query embodiment of the method one provided in an embodiment of the present invention；
Fig. 2 is the flow diagram of data query embodiment of the method two provided in an embodiment of the present invention；
Fig. 3 is the flow diagram of one example of data query method provided in an embodiment of the present invention；
Fig. 4 is the structural schematic diagram of data query device embodiment one provided in an embodiment of the present invention；
Fig. 5 is the structural schematic diagram of data query device embodiment two provided in an embodiment of the present invention；
Fig. 6 is the structural schematic diagram of server example provided in an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
Under the data management scene of large enterprise, a variety of different databases can be used according to specific use demand
System, such as: MySQL, Hive, HBase, Oracle, Elasticsearch, MongoDB etc., according to pervious scheme, exploitation
The learning cost that personnel need to learn these system usage modes is higher, and inconvenient using configuration, if according to background technique
Another scheme of middle offer goes to realize, then the problem lower there is also data system scalability.For those problems, this patent
A kind of data query method is proposed, to solve the above problems.
Noun involved in this case etc. is explained below:
Structured query language (Structured Query Language, SQL): SQL is a kind of specific purpose programming language
Speech is used for administrative relationships data base management system (Relational Database Management System, RDBMS), or
Stream process is carried out in relationship DATA STREAM MANAGE SYSTEM (RDSMS).
Abstract syntax tree (Abstract Syntax Tree, AST) or abbreviation syntax tree (Syntax tree), are source generations
A kind of abstract representation of code syntactic structure.It shows the syntactic structure of programming language, each node on tree in tree-shaped form
All indicate one of source code structure.
Metadata (English: metadata), also known as annotation data, broker data, relaying data, after set data etc., to retouch
State the data of other data informations.There are three types of different types of metadata, be respectively narrate metadata, structural metadata and
Administrative metadata.
HBase is the non-relational distributed data base (NoSQL) of an open source.
Hive is a Tool for Data Warehouse based on Hadoop, the data file of structuring can be mapped as a number
According to library table, and simple SQL query function is provided, SQL statement can be converted to MapReduce task and run.
MySQL is the relational database management system of an open source code,
Oracle Database also known as Oracle RDBMS or abbreviation Oracle.It is a relationship of Oracle
Data base management system.
Based on above content, data query method provided by the present application is described in detail below.
When user needs to carry out data query, inquiry can be sent by the user end to server being mounted in terminal device
Request, server get query result according to query scheme provided by the present application and return to according to the inquiry request received
Client, completes the query process of data, and data query method provided by the present application is mainly to be applied to the realization of server side
Scheme.
Fig. 1 is the flow diagram of data query embodiment of the method one provided in an embodiment of the present invention, as shown in Figure 1, should
Data query method is applied in the server for carrying out data service, specifically includes the following steps:
S101: the first SQL query statement that client is sent is received.
In this step, when needing to carry out the inquiry of data, server opening API, client passes through submission
The mode of http request submits SQL query statement, i.e. user can send SQL query statement by user end to server, right
For server, then can receive the first SQL query statement that different client is sent, be respectively processed, it is subsequent with
For the first SQL query statement that one of client is sent, this programme is illustrated.In the SQL query statement at least
Carry the information of the data for the inquiry that user needs, such as table name.
S102: the first SQL query statement is parsed, and obtains corresponding first AST.
In this step, server is after receiving first SQL query statement, certainly to first SQL query statement
It is dynamic to be parsed, the form of abstract syntax tree is parsed into get corresponding first AST is arrived, and includes being inquired in the AST
Table name where data, server can be associated with corresponding metadata information according to the table name.
S103: according to the first AST, the first metadata information of the first data of the first SQL query statement inquiry is obtained.
The step in the specific implementation, server needs first table name is extracted from the first AST, this first table name
What is indicated is table name belonging to the data to be inquired of the first SQL query statement, then according to first table name and table name and member
Corresponding relationship between data information obtains first metadata information.
S104: query result is obtained according to the first metadata information, and query result is returned into client.
In above-mentioned two step, server is after getting corresponding first AST of the first SQL query statement, this
It include the information of the data to be inquired in one AST, for example including the table name where the data for needing to inquire, server can root
It is associated with corresponding metadata information, that is, the first above-mentioned metadata information according to the table name, is then believed according to the metadata
It ceases and is inquired to corresponding data source, obtain query result and return to client, complete the query result of data.
The program one kind in the specific implementation, in query process, server is needed according to obtained table name, matching pair
The metadata information answered, if matching less than corresponding metadata information, directly returns to the result of inquiry failure to client.
Data query method provided in this embodiment, user have sent SQL query statement by user end to server,
It after server receives the SQL query statement, is parsed to obtain corresponding AST first, inquiry is then obtained from AST
The metadata information of data gets query result from each different database according to metadata information, is then returned to
Client is realized while being supported the function of being inquired the multi-data source of data system, also do not limit what database was stored
Content facilitates user and carries out data query on multi-data source, and set expandability is preferable.
On the basis of the above embodiments, data query method provided by the present application is most evidences based on SQL resolver
Source querying method needs to parse the SQL query statement received, then by way of associated metadata information
The inquiry of data is carried out in different data sources, therefore then needs to store metadata information in local.In the specific of the program
In realization, server needs to be scanned associated multiple data sources, the metadata in each data source is scanned, by every number
Storage is synchronized according to the relevant information in source, i.e., for the operation such as the change of the data in data source, modification, newly-increased, deletion
Information afterwards can also be synchronized to local stored.In general, each data source can be scanned according to predetermined period to information
It synchronizes.
For in the information that stores in the server, that is, data source types are included at least in metadata information, number
According to library name and table name etc..After server has determined table name corresponding with SQL query statement, by metadata information
Information, by traversal queries, acquisition is the corresponding relationship between the table name and metadata information obtained in advance, it is available with should
The corresponding metadata information of SQL query statement.After getting metadata information, data can be carried out according to following process and looked into
It askes:
Fig. 2 is the flow diagram of data query embodiment of the method two provided in an embodiment of the present invention, as shown in Fig. 2, Fig. 1
In step S104 according to the first metadata information obtain query result specifically include:
S1041: according to the first metadata information, the first data source is obtained.
According to the data source stored in the first metadata information, the corresponding relationship between table name, the first SQL above-mentioned is obtained
Data source where the data of query statement inquiry, that is, the first above-mentioned data source.
S1042: it carries out the first SQL query statement being routed to the first data source by SQL router.
S1043: query result is obtained from the first data source query, query result includes the first data.
In above-mentioned steps, the first SQL query statement is directly then passed through into SQL router and is sent to first data source
It is inquired, including at least the first SQL query statement received to obtain subsequent query result, in the query result will look into
The first data ask.
After completing the above process and getting query result, client, a kind of tool can be returned to by open API
In the implementation of body, which can be packaged into unified JSON by server or other formats return to inquiry
User, with no restrictions to this this programme.
On the basis of above-mentioned each embodiment, below by a specific implementation to this based on the more of SQL resolver
The data source query method carried out in a data source is illustrated.
Fig. 3 is the flow diagram of one example of data query method provided in an embodiment of the present invention, as shown in figure 3, in Fig. 3
It shows server internal and realizes the component and process of the data query method, as shown in figure 3, being included at least in the server
SQL resolver, SQL router are able to carry out the memory space being locally stored, query interface, metadata scan timer.It is complete
The data source of external client and multiple and different types is also related to during whole data query, is shown in Fig. 3
Several data sources of MySQL, Hive, HBase, Oracle, Elasticsearch, MongoDB, this programme do not limit data source
Type can be extended according to demand in concrete application.The specific implementation of the program the following steps are included:
1) metadata of each data source, is scanned, metadata relevant information that is synchronous and storing each data source is to locally
Database, storage information includes but is not limited to data source types, database name, table name etc..
2), system initiation task timer, timing execute step (1).Ensured by way of timer if data
The table in source is increased newly, is deleted, and local database can be also updated to.
3), this system opening API, client submit SQL query statement by way of submitting http request, and
Waiting returns the result.
4), system is parsed into AST syntax tree according to the SQL statement of submission automatically, and is extracted AST syntax tree
In table name (table name), and the metadata information of associated storage.
5), according to associated metadata information, the metadata information of the table name is obtained.Obtained table name-data source class
Type-database name-table name match information, if matching less than table name relative to metadata information, directly return inquire
Failure.
6), the metadata information for being matched to previous step, submit to SQL router carry out data source by.
7), SQL statement is routed automatically in specified data source by SQL router, executes inquiry.If specified data
SQL statement, such as HBase data source are not supported in source, then need to provide plug-in unit, realize from SQL statement to the data resource interface
Conversion.Such as: if the first data source does not support SQL statement, the first SQL query statement is converted, is converted to the first number
According to the query statement that source is supported, to guarantee to complete the data query process.
8), data source returns to query result, meanwhile, a unique ID (UUID) is generated, and to the result of this inquiry
It is cached, i.e., the query result is stored, and the query result is identified by first identifier, in order to
The subsequent acquisition of user as a result, reduce secondary query time significantly.
9) query result, is packaged into unified JSON or extended formatting returns to inquiry user.
The technical solution of the application provides a kind of data query method in diversiform data source.It is automatically parsed using SQL,
And the step of associated metadata, the function that SQL query statement is submitted to different data sources automatically is realized, user does not need to be concerned about
The storage form of behind data source, to solve the problems, such as that client needs to be adapted to different data sources.
Fig. 4 is the structural schematic diagram of data query device embodiment one provided in an embodiment of the present invention, as shown in figure 4, should
Data query device 10 includes:
Receiving module 11, for receiving the first SQL query statement of client transmission；
Processing module 12 obtains corresponding first AST for parsing first SQL query statement；
The processing module 12 is also used to obtain the first of the first SQL query statement inquiry according to the first AST
First metadata information of data；
Enquiry module 13, for obtaining query result according to first metadata information；
Sending module 14, for the query result to be returned to the client.
Data query device provided in this embodiment, for realizing the technical solution of aforementioned either method embodiment, in fact
Existing principle is similar with technical effect, and user has sent SQL query statement to data query device by client, looks into the data
It after inquiry device receives the SQL query statement, is parsed to obtain corresponding AST first, inquiry is then obtained from AST
The metadata information of data gets query result from each different database according to metadata information, is then returned to
Client is realized while being supported the function of being inquired the multi-data source of data system, also do not limit what database was stored
Content facilitates user and carries out data query on multi-data source, and set expandability is preferable.
On the basis of the above embodiments, optionally, the AST includes the first table name belonging to first data, institute
Processing module 12 is stated to be specifically used for:
First table name is extracted from the first AST；
According to the corresponding relationship between first table name and table name and metadata information, the first metadata letter is obtained
Breath.
Optionally, the enquiry module 13 is specifically used for:
According to first metadata information, the first data source is obtained；
It carries out first SQL query statement being routed to first data source by SQL router；
The query result is obtained from first data source query, the query result includes first data.
Optionally, the processing module 12 is also used to:
If first data source does not support SQL statement, first SQL query statement is converted, is converted to
The query statement that first data source is supported.
Optionally, the processing module 12 is also used to:
Each data source is scanned according to predetermined period, the corresponding metadata information of each data source being locally stored is carried out
Synchronous, each metadata information includes at least data source types, database name and table name.
The data query device of the offer of any of the above-described implementation, for executing the skill in aforementioned either method embodiment
Art scheme, it is similar that the realization principle and technical effect are similar, and details are not described herein.
Fig. 5 is the structural schematic diagram of data query device embodiment two provided in an embodiment of the present invention, as shown in figure 5, should
Data query device 10 further include:
Memory module 15, for being stored to the query result, and by first identifier to the query result into
Line identifier.
Based on any of the above embodiments, the sending module 14 is specifically used for:
The query result is packaged into JSON and returns to the client.
The data query device of the offer of any of the above-described implementation, for executing the skill in aforementioned either method embodiment
Art scheme, it is similar that the realization principle and technical effect are similar, and details are not described herein.
Fig. 6 is the structural schematic diagram of server example one provided in an embodiment of the present invention, as shown in fig. 6, the present embodiment
The server of offer may include: receiver, memory, processor and transmitter.
Optionally, which can also include bus.Wherein, bus is for realizing the connection between each element.
The memory calls the computer journey of memory storage for storing computer program and data, the processor
Sequence, to execute the technical solution for the data query method that aforementioned either method embodiment provides.
Wherein, it is directly or indirectly electrically connected between memory and processor, to realize the transmission or interaction of data.Example
Such as, these elements can be realized by one or more of communication bus or signal wire be electrically connected between each other, can such as lead to
Cross bus connection.The computer executed instructions for realizing data access control method are stored in memory, including at least one can
The software function module being stored in memory 201 in the form of software or firmware, processor are stored in memory by operation
Interior computer program and module, thereby executing various function application and data processing.
Memory may be, but not limited to, random access memory (Random Access Memory, referred to as: RAM),
Read-only memory (Read Only Memory, referred to as: ROM), programmable read only memory (Programmable Read-Only
Memory, referred to as: PROM), erasable read-only memory (Erasable Programmable Read-Only Memory, letter
Claim: EPROM), electricallyerasable ROM (EEROM) (Electric Erasable Programmable Read-Only Memory,
Referred to as: EEPROM) etc..Wherein, memory is for storing program, and processor executes program after receiving and executing instruction.Into one
Step, software program and module in above-mentioned memory may also include operating system, may include various for management system
The component software of task (such as memory management, storage equipment control, power management etc.) and/or driving, and can be with various hardware
Or component software is in communication with each other, to provide the running environment of other software component.
Processor can be a kind of IC chip, the processing capacity with signal.Above-mentioned processor can be logical
With processor, including central processing unit (Central Processing Unit, referred to as: CPU), network processing unit (Network
Processor, referred to as: NP) etc..It may be implemented or execute disclosed each method, step and the logic in the embodiment of the present invention
Block diagram.General processor can be microprocessor or the processor is also possible to any conventional processor etc..It is appreciated that
The structure of Fig. 6 be only illustrate, can also include than shown in Fig. 6 more perhaps less component or have with shown in Fig. 6 not
Same configuration.Each component shown in Fig. 6 can use hardware and/or software realization.
The embodiment of the present invention also provides a kind of computer readable storage medium, is stored thereon with computer program, computer
The data query method that any of the above-described embodiment of the method provides may be implemented when program is executed by processor.
Computer readable storage medium in the present embodiment can be any usable medium that computer can access, or
It is that can be magnetic comprising data storage devices, usable mediums such as one or more usable mediums integrated server, data centers
Property medium, (for example, floppy disk, hard disk, tape), optical medium (for example, DVD) or semiconductor medium (such as SSD) etc..
Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above-mentioned each method embodiment can lead to
The relevant hardware of program instruction is crossed to complete.Program above-mentioned can be stored in a computer readable storage medium.The journey
When being executed, execution includes the steps that above-mentioned each method embodiment to sequence；And storage medium above-mentioned include: ROM, RAM, magnetic disk or
The various media that can store program code such as person's CD.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme.
Claims (16)
1. a kind of data query method, which is characterized in that the described method includes:
Receive the first structure query language SQL query statement that client is sent；
First SQL query statement is parsed, corresponding first abstract syntax tree AST is obtained；
According to the first AST, the first metadata information of the first data of the first SQL query statement inquiry is obtained；
Query result is obtained according to first metadata information, and the query result is returned into the client.
2. the method according to claim 1, wherein the AST includes the first table belonging to first data
Name, it is described that first metadata information of the first data of the first SQL query statement inquiry is obtained according to the first AST,
Include:
First table name is extracted from the first AST；
According to the corresponding relationship between first table name and table name and metadata information, first metadata information is obtained.
3. the method according to claim 1, wherein described obtain inquiry knot according to first metadata information
Fruit, comprising:
According to first metadata information, the first data source is obtained；
It carries out first SQL query statement being routed to first data source by SQL router；
The query result is obtained from first data source query, the query result includes first data.
4. according to the method described in claim 3, it is characterized in that, the method also includes:
If first data source does not support SQL statement, first SQL query statement is converted, is converted to described
The query statement that first data source is supported.
5. method according to any one of claims 1 to 4, which is characterized in that the method also includes:
Each data source is scanned according to predetermined period, the corresponding metadata information of each data source being locally stored is carried out same
Step, each metadata information include at least data source types, database name and table name.
6. method according to any one of claims 1 to 4, which is characterized in that the method also includes:
The query result is stored, and the query result is identified by first identifier.
7. method according to any one of claims 1 to 4, which is characterized in that it is described will the query result return described in
Client, comprising:
The query result is packaged into JSON and returns to the client.
8. a kind of data query device characterized by comprising
Receiving module, for receiving the first structure query language SQL query statement of client transmission；
Processing module obtains corresponding first abstract syntax tree AST for parsing first SQL query statement；
The processing module is also used to obtain the first data of the first SQL query statement inquiry according to the first AST
First metadata information；
Enquiry module, for obtaining query result according to first metadata information；
Sending module, for the query result to be returned to the client.
9. device according to claim 8, which is characterized in that the AST includes the first table belonging to first data
Name, the processing module are specifically used for:
First table name is extracted from the first AST；
According to the corresponding relationship between first table name and table name and metadata information, first metadata information is obtained.
10. device according to claim 8, which is characterized in that the enquiry module is specifically used for:
According to first metadata information, the first data source is obtained；
It carries out first SQL query statement being routed to first data source by SQL router；
The query result is obtained from first data source query, the query result includes first data.
11. device according to claim 10, which is characterized in that the processing module is also used to:
If first data source does not support SQL statement, first SQL query statement is converted, is converted to described
The query statement that first data source is supported.
12. device according to any one of claims 8 to 11, which is characterized in that the processing module is also used to:
Each data source is scanned according to predetermined period, the corresponding metadata information of each data source being locally stored is carried out same
Step, each metadata information include at least data source types, database name and table name.
13. device according to any one of claims 8 to 11, which is characterized in that described device further include:
Memory module is identified the query result for storing to the query result, and by first identifier.
14. device according to any one of claims 8 to 11, which is characterized in that the sending module is specifically used for:
The query result is packaged into JSON and returns to the client.
15. a kind of server characterized by comprising
Receiver, processor, memory and transmitter；
Memory is for storing computer program and data, and the processor calls the computer program of memory storage, to hold
The described in any item data query methods of row claim 1 to 7.
16. a kind of computer readable storage medium, which is characterized in that the computer readable storage medium includes computer journey
Sequence, the computer program require 1 to 7 described in any item data query sides for perform claim when being executed by processor
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910298551.3A CN110032575A (en) | 2019-04-15 | 2019-04-15 | Data query method, apparatus, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910298551.3A CN110032575A (en) | 2019-04-15 | 2019-04-15 | Data query method, apparatus, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110032575A true CN110032575A (en) | 2019-07-19 |
Family
ID=67238367
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910298551.3A Pending CN110032575A (en) | 2019-04-15 | 2019-04-15 | Data query method, apparatus, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110032575A (en) |
Cited By (41)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110389968A (en) * | 2019-07-31 | 2019-10-29 | 中国工商银行股份有限公司 | Aggregate query method, aggregate query device, equipment and medium |
| CN110399388A (en) * | 2019-07-29 | 2019-11-01 | 中国工商银行股份有限公司 | Data query method, system and equipment |
| CN110633281A (en) * | 2019-09-12 | 2019-12-31 | 北京百度网讯科技有限公司 | Method and device for processing multi-type data sources |
| CN110633292A (en) * | 2019-09-19 | 2019-12-31 | 上海依图网络科技有限公司 | Query method, device, medium, equipment and system for heterogeneous database |
| CN110688544A (en) * | 2019-10-17 | 2020-01-14 | 北京锐安科技有限公司 | Method, device and storage medium for querying database |
| CN110781212A (en) * | 2019-09-20 | 2020-02-11 | 成都四方伟业软件股份有限公司 | Method, assembly and system for efficiently calculating and accessing SQL |
| CN110909386A (en) * | 2019-11-21 | 2020-03-24 | 福建南威软件有限公司 | Unified authorization access method and system for multiple data sources |
| CN110990425A (en) * | 2019-11-28 | 2020-04-10 | 中科驭数（北京）科技有限公司 | JDBC driving method and device of hardware database |
| CN111078961A (en) * | 2019-12-24 | 2020-04-28 | 用友网络科技股份有限公司 | Multi-data source query driving system, method, device and storage medium |
| CN111090667A (en) * | 2019-12-04 | 2020-05-01 | 北京奇艺世纪科技有限公司 | Data processing method, device, server and storage medium |
| CN111104421A (en) * | 2019-12-31 | 2020-05-05 | 新奥数能科技有限公司 | Data query method and device based on data interface standard configuration |
| CN111125472A (en) * | 2019-12-27 | 2020-05-08 | 北京明略软件系统有限公司 | Result backtracking method and device, electronic equipment and storage medium |
| CN111198901A (en) * | 2020-01-06 | 2020-05-26 | 北京邮电大学 | Method and system for opening unified data service based on multiple data sources |
| CN111209326A (en) * | 2020-01-02 | 2020-05-29 | 广州虎牙科技有限公司 | Data query method and device, electronic equipment and storage medium |
| CN111221804A (en) * | 2019-12-27 | 2020-06-02 | 北京健康之家科技有限公司 | Method, device and storage medium for preventing data from being unauthorized based on abstract syntax tree |
| CN111581231A (en) * | 2020-04-20 | 2020-08-25 | 北京明略软件系统有限公司 | Query method and device based on heterogeneous database |
| CN111767311A (en) * | 2020-06-05 | 2020-10-13 | 浙江大搜车软件技术有限公司 | Variable request method, system, computer device and computer readable storage medium |
| CN112069201A (en) * | 2020-09-04 | 2020-12-11 | 北京百度网讯科技有限公司 | Target data acquisition method and device |
| CN112115122A (en) * | 2020-09-18 | 2020-12-22 | 北京金山云网络技术有限公司 | Data access method and device and electronic equipment |
| CN112347108A (en) * | 2020-11-26 | 2021-02-09 | 上海天旦网络科技发展有限公司 | Data query method and system suitable for hybrid backend |
| CN112351099A (en) * | 2020-11-06 | 2021-02-09 | 北京金山云网络技术有限公司 | Data access method and device and server |
| CN112416964A (en) * | 2020-11-17 | 2021-02-26 | 深圳依时货拉拉科技有限公司 | Data processing method, device and system, computer equipment and computer readable storage medium |
| CN112434189A (en) * | 2020-12-02 | 2021-03-02 | 新华三大数据技术有限公司 | Data query method, device and equipment |
| CN112487014A (en) * | 2020-11-27 | 2021-03-12 | 北京金山云网络技术有限公司 | SQL data processing method and device and background server |
| CN112541001A (en) * | 2019-09-23 | 2021-03-23 | 北京国双科技有限公司 | Data query method, device, storage medium and equipment |
| CN112667663A (en) * | 2020-12-28 | 2021-04-16 | 北京明略软件系统有限公司 | Data query method and system |
| CN113064914A (en) * | 2021-04-22 | 2021-07-02 | 中国工商银行股份有限公司 | Data extraction method and device |
| CN113094387A (en) * | 2021-04-08 | 2021-07-09 | 杭州数梦工场科技有限公司 | Data query method and device, electronic equipment and machine-readable storage medium |
| WO2021139426A1 (en) * | 2020-01-06 | 2021-07-15 | 平安科技（深圳）有限公司 | Method, device and apparatus for querying data in database, and storage medium |
| CN113127558A (en) * | 2019-12-31 | 2021-07-16 | 北京懿医云科技有限公司 | Metadata synchronization method, system, device and storage medium |
| CN113157657A (en) * | 2021-05-18 | 2021-07-23 | 山东健康医疗大数据有限公司 | Method and system for timing synchronization of multiple data source databases |
| CN113590652A (en) * | 2021-09-30 | 2021-11-02 | 阿里云计算有限公司 | Data storage system, data processing method, device and storage medium |
| CN113760947A (en) * | 2020-12-24 | 2021-12-07 | 北京京东尚科信息技术有限公司 | Data center, data processing method, device, equipment and storage medium |
| CN113901071A (en) * | 2020-06-22 | 2022-01-07 | 浙江宇视科技有限公司 | A storage medium-oriented query method, device, medium, device and system |
| CN114003231A (en) * | 2021-09-28 | 2022-02-01 | 厦门国际银行股份有限公司 | SQL syntax parse tree optimization method and system |
| CN114201507A (en) * | 2021-12-14 | 2022-03-18 | 平安养老保险股份有限公司 | Method, device, equipment and storage medium for log query based on ElasticSearch |
| CN114328574A (en) * | 2021-11-29 | 2022-04-12 | 上海欣兆阳信息科技有限公司 | A data query method, apparatus, electronic device, and computer-readable storage medium |
| CN114490842A (en) * | 2021-12-28 | 2022-05-13 | 航天科工智慧产业发展有限公司 | Interface data query method and data query engine for multi-source data |
| CN114692208A (en) * | 2022-05-31 | 2022-07-01 | 中建电子商务有限责任公司 | Processing method of data query service authority |
| CN114706625A (en) * | 2022-03-29 | 2022-07-05 | 智业软件股份有限公司 | Method, device and storage medium for constructing patient information global query plug-in |
| CN117453731A (en) * | 2023-12-22 | 2024-01-26 | 北京宇信科技集团股份有限公司 | Multi-source data query system and multi-source data query method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105989150A (en) * | 2015-03-02 | 2016-10-05 | 中国移动通信集团四川有限公司 | Data query method and device based on big data environment |
| CN106649630A (en) * | 2016-12-07 | 2017-05-10 | 乐视控股（北京）有限公司 | Data query method and device |
| CN106682147A (en) * | 2016-12-22 | 2017-05-17 | 北京锐安科技有限公司 | Mass data based query method and device |
| US20180025061A1 (en) * | 2015-03-26 | 2018-01-25 | International Business Machines Corporation | Bootstrapping the data lake and glossaries with 'dataset joins' metadata from existing application patterns |
-
2019
- 2019-04-15 CN CN201910298551.3A patent/CN110032575A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105989150A (en) * | 2015-03-02 | 2016-10-05 | 中国移动通信集团四川有限公司 | Data query method and device based on big data environment |
| US20180025061A1 (en) * | 2015-03-26 | 2018-01-25 | International Business Machines Corporation | Bootstrapping the data lake and glossaries with 'dataset joins' metadata from existing application patterns |
| CN106649630A (en) * | 2016-12-07 | 2017-05-10 | 乐视控股（北京）有限公司 | Data query method and device |
| CN106682147A (en) * | 2016-12-22 | 2017-05-17 | 北京锐安科技有限公司 | Mass data based query method and device |
Cited By (48)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110399388A (en) * | 2019-07-29 | 2019-11-01 | 中国工商银行股份有限公司 | Data query method, system and equipment |
| CN110389968A (en) * | 2019-07-31 | 2019-10-29 | 中国工商银行股份有限公司 | Aggregate query method, aggregate query device, equipment and medium |
| CN110633281A (en) * | 2019-09-12 | 2019-12-31 | 北京百度网讯科技有限公司 | Method and device for processing multi-type data sources |
| CN110633292A (en) * | 2019-09-19 | 2019-12-31 | 上海依图网络科技有限公司 | Query method, device, medium, equipment and system for heterogeneous database |
| CN110633292B (en) * | 2019-09-19 | 2022-06-21 | 上海依图网络科技有限公司 | Query method, device, medium, equipment and system for heterogeneous database |
| CN110781212A (en) * | 2019-09-20 | 2020-02-11 | 成都四方伟业软件股份有限公司 | Method, assembly and system for efficiently calculating and accessing SQL |
| CN112541001A (en) * | 2019-09-23 | 2021-03-23 | 北京国双科技有限公司 | Data query method, device, storage medium and equipment |
| CN110688544A (en) * | 2019-10-17 | 2020-01-14 | 北京锐安科技有限公司 | Method, device and storage medium for querying database |
| CN110909386A (en) * | 2019-11-21 | 2020-03-24 | 福建南威软件有限公司 | Unified authorization access method and system for multiple data sources |
| CN110990425B (en) * | 2019-11-28 | 2020-12-25 | 中科驭数（北京）科技有限公司 | JDBC driving method and device of hardware database |
| CN110990425A (en) * | 2019-11-28 | 2020-04-10 | 中科驭数（北京）科技有限公司 | JDBC driving method and device of hardware database |
| CN111090667A (en) * | 2019-12-04 | 2020-05-01 | 北京奇艺世纪科技有限公司 | Data processing method, device, server and storage medium |
| CN111078961A (en) * | 2019-12-24 | 2020-04-28 | 用友网络科技股份有限公司 | Multi-data source query driving system, method, device and storage medium |
| CN111078961B (en) * | 2019-12-24 | 2023-09-15 | 用友网络科技股份有限公司 | Multi-data source query driving system, method, device and storage medium |
| CN111221804A (en) * | 2019-12-27 | 2020-06-02 | 北京健康之家科技有限公司 | Method, device and storage medium for preventing data from being unauthorized based on abstract syntax tree |
| CN111125472A (en) * | 2019-12-27 | 2020-05-08 | 北京明略软件系统有限公司 | Result backtracking method and device, electronic equipment and storage medium |
| CN111125472B (en) * | 2019-12-27 | 2023-05-02 | 北京明略软件系统有限公司 | Result backtracking method and device, electronic equipment and storage medium |
| CN113127558A (en) * | 2019-12-31 | 2021-07-16 | 北京懿医云科技有限公司 | Metadata synchronization method, system, device and storage medium |
| CN111104421A (en) * | 2019-12-31 | 2020-05-05 | 新奥数能科技有限公司 | Data query method and device based on data interface standard configuration |
| CN111209326A (en) * | 2020-01-02 | 2020-05-29 | 广州虎牙科技有限公司 | Data query method and device, electronic equipment and storage medium |
| WO2021139426A1 (en) * | 2020-01-06 | 2021-07-15 | 平安科技（深圳）有限公司 | Method, device and apparatus for querying data in database, and storage medium |
| CN111198901A (en) * | 2020-01-06 | 2020-05-26 | 北京邮电大学 | Method and system for opening unified data service based on multiple data sources |
| CN111198901B (en) * | 2020-01-06 | 2023-03-10 | 北京邮电大学 | Unified data service opening method and system based on multiple data sources |
| CN111581231A (en) * | 2020-04-20 | 2020-08-25 | 北京明略软件系统有限公司 | Query method and device based on heterogeneous database |
| CN111767311A (en) * | 2020-06-05 | 2020-10-13 | 浙江大搜车软件技术有限公司 | Variable request method, system, computer device and computer readable storage medium |
| CN113901071A (en) * | 2020-06-22 | 2022-01-07 | 浙江宇视科技有限公司 | A storage medium-oriented query method, device, medium, device and system |
| CN112069201A (en) * | 2020-09-04 | 2020-12-11 | 北京百度网讯科技有限公司 | Target data acquisition method and device |
| EP3816815A1 (en) * | 2020-09-04 | 2021-05-05 | Beijing Baidu Netcom Science Technology Co., Ltd. | Target data obtaining method and apparatus |
| CN112115122A (en) * | 2020-09-18 | 2020-12-22 | 北京金山云网络技术有限公司 | Data access method and device and electronic equipment |
| CN112115122B (en) * | 2020-09-18 | 2024-08-30 | 北京金山云网络技术有限公司 | Data access method and device and electronic equipment |
| CN112351099A (en) * | 2020-11-06 | 2021-02-09 | 北京金山云网络技术有限公司 | Data access method and device and server |
| CN112416964A (en) * | 2020-11-17 | 2021-02-26 | 深圳依时货拉拉科技有限公司 | Data processing method, device and system, computer equipment and computer readable storage medium |
| CN112347108A (en) * | 2020-11-26 | 2021-02-09 | 上海天旦网络科技发展有限公司 | Data query method and system suitable for hybrid backend |
| CN112487014A (en) * | 2020-11-27 | 2021-03-12 | 北京金山云网络技术有限公司 | SQL data processing method and device and background server |
| CN112434189A (en) * | 2020-12-02 | 2021-03-02 | 新华三大数据技术有限公司 | Data query method, device and equipment |
| CN113760947A (en) * | 2020-12-24 | 2021-12-07 | 北京京东尚科信息技术有限公司 | Data center, data processing method, device, equipment and storage medium |
| CN112667663A (en) * | 2020-12-28 | 2021-04-16 | 北京明略软件系统有限公司 | Data query method and system |
| CN113094387A (en) * | 2021-04-08 | 2021-07-09 | 杭州数梦工场科技有限公司 | Data query method and device, electronic equipment and machine-readable storage medium |
| CN113064914A (en) * | 2021-04-22 | 2021-07-02 | 中国工商银行股份有限公司 | Data extraction method and device |
| CN113157657A (en) * | 2021-05-18 | 2021-07-23 | 山东健康医疗大数据有限公司 | Method and system for timing synchronization of multiple data source databases |
| CN114003231A (en) * | 2021-09-28 | 2022-02-01 | 厦门国际银行股份有限公司 | SQL syntax parse tree optimization method and system |
| CN113590652A (en) * | 2021-09-30 | 2021-11-02 | 阿里云计算有限公司 | Data storage system, data processing method, device and storage medium |
| CN114328574A (en) * | 2021-11-29 | 2022-04-12 | 上海欣兆阳信息科技有限公司 | A data query method, apparatus, electronic device, and computer-readable storage medium |
| CN114201507A (en) * | 2021-12-14 | 2022-03-18 | 平安养老保险股份有限公司 | Method, device, equipment and storage medium for log query based on ElasticSearch |
| CN114490842A (en) * | 2021-12-28 | 2022-05-13 | 航天科工智慧产业发展有限公司 | Interface data query method and data query engine for multi-source data |
| CN114706625A (en) * | 2022-03-29 | 2022-07-05 | 智业软件股份有限公司 | Method, device and storage medium for constructing patient information global query plug-in |
| CN114692208A (en) * | 2022-05-31 | 2022-07-01 | 中建电子商务有限责任公司 | Processing method of data query service authority |
| CN117453731A (en) * | 2023-12-22 | 2024-01-26 | 北京宇信科技集团股份有限公司 | Multi-source data query system and multi-source data query method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110032575A (en) | Data query method, apparatus, equipment and storage medium | |
| US11880721B2 (en) | Processing a query having calls to multiple data sources | |
| CN110032604B (en) | Data storage device, translation device and database access method | |
| US10558656B2 (en) | Optimizing write operations in object schema-based application programming interfaces (APIS) | |
| US10387496B2 (en) | Storing graph data in a relational database | |
| US20170346875A1 (en) | Optimizing read operations in object schema-based application programming interfaces (apis) | |
| CN108133007A (en) | A kind of method of data synchronization and system | |
| CN110866029B (en) | sql statement construction method, device, server and readable storage medium | |
| CN109408689A (en) | Data capture method, device, system and electronic equipment | |
| EP3987406A1 (en) | Graph processing system | |
| US11704327B2 (en) | Querying distributed databases | |
| CN113886485A (en) | Data processing method, apparatus, electronic device, system and storage medium | |
| CN113760961A (en) | Data query method and device | |
| US12117999B2 (en) | Masking shard operations in distributed database systems | |
| US11573960B2 (en) | Application-based query transformations | |
| CN105446981B (en) | Map of website generation method, access method and device | |
| US11403313B2 (en) | Dynamic visualization of application and infrastructure components with layers | |
| CN114238459A (en) | A method, device and system for integrated management of heterogeneous data sources | |
| US20240256545A1 (en) | Runtime systems query coordinator | |
| CN115344614B (en) | Data processing method, device, storage medium and electronic device | |
| CN111126961A (en) | Complex product full life cycle digital mainline service system | |
| CN112988778A (en) | Method and device for processing database query script | |
| WO2023164294A1 (en) | Query splitter for an inverted index datastore | |
| US11727022B2 (en) | Generating a global delta in distributed databases | |
| CN113760949B (en) | Data query method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190719 |
|
| RJ01 | Rejection of invention patent application after publication |