[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2019184739A1 - 一种数据查询方法、装置及设备 - Google Patents

一种数据查询方法、装置及设备 Download PDF

Info

Publication number
WO2019184739A1
WO2019184739A1 PCT/CN2019/078418 CN2019078418W WO2019184739A1 WO 2019184739 A1 WO2019184739 A1 WO 2019184739A1 CN 2019078418 W CN2019078418 W CN 2019078418W WO 2019184739 A1 WO2019184739 A1 WO 2019184739A1
Authority
WO
WIPO (PCT)
Prior art keywords
query
resource
query request
feature information
data
Prior art date
Application number
PCT/CN2019/078418
Other languages
English (en)
French (fr)
Inventor
周祥
李冰
赵永春
温绍锦
Original Assignee
阿里巴巴集团控股有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司 filed Critical 阿里巴巴集团控股有限公司
Priority to EP19775405.4A priority Critical patent/EP3779688A4/en
Priority to JP2020552239A priority patent/JP7511477B2/ja
Publication of WO2019184739A1 publication Critical patent/WO2019184739A1/zh
Priority to US17/035,276 priority patent/US11556541B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24553—Query execution of query operations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2471—Distributed queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/248—Presentation of query results
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5011—Pool
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals

Definitions

  • the present application relates to the field of Internet technologies, and in particular, to a data query method, apparatus, and device.
  • Open Analytics is used to provide users with serverless query analysis services. It can analyze and query arbitrary data in any dimension, support high concurrency, low latency (millisecond response), real-time online. Analysis, massive data query and other functions.
  • the open analysis service system may include a data source and a computing node.
  • the data source is used to store a large amount of data. After receiving the query request, the computing node queries the data source for the data corresponding to the query request.
  • the computing node may receive multiple query requests in a short time (ie, the number of concurrent calls is high), that is, in a short time
  • the processing of multiple query requests causes an abnormality in the CPU (Central Processing Unit) resources, memory resources, network bandwidth, etc., resulting in query timeout or query failure.
  • CPU Central Processing Unit
  • the application provides a data query method, the method comprising:
  • the data corresponding to the query request is queried by the computing node.
  • the application provides a data query method, the method comprising:
  • Decoding according to the feature information of the received query request, the received query request into at least one allocation group; wherein different allocation groups correspond to different sub-resource pools;
  • Data corresponding to the query request in the allocation group is queried through a compute node in the child resource pool.
  • the application provides a data query device, and the device includes:
  • Obtaining a module configured to obtain a resource overhead according to the feature information of the received query request
  • a query module configured to query, by the computing node, data corresponding to the query request.
  • the application provides a data query device, and the device includes:
  • a dividing module configured to divide the received query request into at least one allocation group according to the feature information of the received query request; wherein different allocation groups correspond to different sub-resource pools;
  • an obtaining module configured to obtain, according to the feature information of the query request in the allocation group, a resource overhead of the allocation group
  • a processing module configured to dynamically adjust a computing node in the sub-resource pool according to the resource cost of the allocation group and the computing node resource of the sub-resource pool corresponding to the allocation group;
  • a query module configured to query, by the computing node in the sub-resource pool, data corresponding to the query request in the allocation group.
  • the present application provides a data query device, including: a processor, configured to obtain a resource overhead according to the feature information of the received query request; dynamically adjust a computing node in the resource pool according to the resource cost and the computing node resource; Query data corresponding to the query request.
  • the present application provides a data query device, including: a processor, configured to divide a received query request into at least one allocation group according to the feature information of the received query request; wherein different allocation groups correspond to different sub-resources a pool; obtaining a resource cost of the allocation group according to the feature information of the query request in the allocation group; dynamically adjusting the sub-resource according to the resource cost of the allocation group and the computing node resource of the sub-resource pool corresponding to the allocation group a computing node in the pool; querying, by the computing node in the child resource pool, data corresponding to the query request in the allocation group.
  • the resource overhead may be obtained according to the feature information of the received query request, and the computing node in the resource pool may be dynamically adjusted according to the resource cost and the computing node resource, so that the computing node in the resource pool can Processing all the query requests received, more effectively improving the processing efficiency and resource utilization of the computing nodes, can enable the computing nodes to process multiple query requests in parallel and improve the utilization of CPU resources, memory resources, and network bandwidth resources. Therefore, from the perspective of the overall computing resources and user query load to achieve a better effect, improve user experience.
  • each computing node can provide a serverless query analysis service for the user, so that the user does not need to perceive the server or the service instance, and only needs to perceive the service provided by the cloud service.
  • the cloud service Based on the cloud service, the user only needs to input the SQL query request, and the data can be queried and analyzed by the computing node in the data source, and the business analysis tools and applications can be seamlessly integrated. Intelligent analysis and automatic adjustment of resources can be used to more effectively improve resource utilization and cost performance of cloud databases and cloud data analysis service clusters.
  • FIG. 1 is a schematic structural diagram of a system in an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a system in another embodiment of the present application.
  • FIG. 5 is a structural diagram of a data query apparatus in an embodiment of the present application.
  • FIG. 6 is a structural diagram of a data query device in another embodiment of the present application.
  • first, second, third, etc. may be used to describe various information in the embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information without departing from the scope of the present application.
  • second information may also be referred to as the first information.
  • word “if” may be interpreted to mean “at time” or "when” or "in response to determination.”
  • the embodiment of the present application provides a data query method, which can be applied to a client, a load balancing device, a front node (also referred to as a front end server), and a compute node (also referred to as a computing server). And systems of data sources, such as systems for implementing open analytics services. Of course, other servers, such as a resource scheduling server, may also be included, and no limitation is imposed on this.
  • FIG. 1 a schematic diagram of an application scenario in the embodiment of the present application includes one or more front-end nodes in a resource pool of a front-end node, and FIG. 1 takes three front-end nodes as an example.
  • FIG. 1 takes 5 computing nodes as an example.
  • the number of the embodiments is for the computing node to expand or shrink.
  • the client is used to query data from the data source, such as an application (application) included in a terminal device (such as a PC (Personal Computer), a notebook computer, a mobile terminal, etc.), or may be a terminal device.
  • a terminal device such as a PC (Personal Computer), a notebook computer, a mobile terminal, etc.
  • the included browser does not impose specific restrictions on the type of this client.
  • the load balancing device is configured to perform load balancing on the query request. For example, after receiving a large number of query requests, the load can be load-balanced to each front-end node, and the process is not limited.
  • the data source is used to store various types of data, and can provide data stored in the data source to the client.
  • data stored in the data source there is no limitation in the embodiment of the present application, such as user data, product data, map data, video data, image data, audio data, and the like.
  • the front-end node is configured to receive a query request sent by the client, perform a SQL (Structured Query Language) parsing on the query request, generate a query request by using the SQL parsing result, and send the query request to the query request.
  • a compute node requesting data corresponding to the query request.
  • the front-end node is further configured to receive data returned by the computing node and send the data to the client.
  • the computing node is configured to receive a query request sent by the front-end node, and use the query request to read data corresponding to the query request from the data source, and the reading process is not limited, and the data is sent to the front end. node.
  • a computing node receives a large number of query requests in a short period of time (that is, the number of concurrent calls is high)
  • the computing node needs to process a large number of query requests in a short period of time, resulting in abnormalities in CPU resources, memory resources, and network bandwidth. , causing the query to time out or the query to fail.
  • the computing node in the resource pool can be dynamically adjusted, that is, when there are a large number of query requests, the number of computing nodes in the resource pool can be increased, and the number of query requests of each computing node can be reduced.
  • a computing node can avoid processing a large number of query requests in a short period of time, more effectively improve the processing efficiency and resource utilization of the computing node, and reduce the occupation of CPU resources, memory resources, and network bandwidth, thereby improving processing performance and avoiding customers.
  • the end query times out or fails, improving the user experience.
  • FIG. 2 it is a schematic flowchart of a data query method according to an embodiment of the present application.
  • the method may be applied to a data query device, where the data query device may be the load balancing device in FIG.
  • the front-end node or the resource scheduling server is not limited.
  • the method is applied to the front-end node.
  • the method may include the following steps:
  • Step 201 Obtain resource overhead according to the feature information of the received query request.
  • the resource overhead may be obtained according to the feature information of the query request received in the preset time window.
  • Step 202 Dynamically adjust a computing node in the resource pool according to the resource overhead and the computing node resource.
  • Step 203 Query, by the computing node in the resource pool, data corresponding to the query request.
  • the above-described execution order is only an example given for convenience of description. In an actual application, the execution order between the steps may also be changed, and the order of execution is not limited. Moreover, in other embodiments, the steps of the respective methods are not necessarily performed in the order shown and described herein, and the methods may include more or less steps than those described in this specification. In addition, the individual steps described in this specification may be decomposed into a plurality of steps for description in other embodiments; the various steps described in the present specification may be combined into a single step for description in other embodiments.
  • the query request may be sent.
  • the load balancing device may send the query request to the front-end node, and the front-end node receives the query request.
  • This query request can be stored in a query queue (Query Queue).
  • the front end node may be configured with a preset time window, and the time of the preset time window may be configured according to experience, such as 3 seconds. Based on this, the front-end node can determine all the query requests stored in the query queue in the preset time window as the query requests received within the preset time window, such as 100 query requests.
  • the feature information of each query request may also be acquired first, and the feature information may include but is not limited to one or any combination of the following: concurrency, query Complexity, amount of query data, query time, resource occupancy.
  • Concurrency which is the number of query requests received within the preset time window, such as 100.
  • the query complexity that is, the complexity of executing the query request
  • the query complexity is usually a value.
  • the query complexity, CPU resource usage, memory resource usage, and network bandwidth usage are normalized to obtain the query complexity. For example, if the query request 1 is executed, a large amount of CPU resources, memory resources, and network bandwidth are required, and the query time is long, and the query complexity of the query request 1 is high. If the query request 2 is executed, a small amount of CPU resources, memory resources, and network bandwidth are required, and the query time is short, and the query complexity of the query request 2 is low.
  • the query complexity is the same or similar. Therefore, the correspondence between the query keyword and the complexity value can be obtained, and the correspondence between the query keyword and the complexity value is recorded in the first mapping table. relationship. For example, assuming that both the query request 1 and the query request 2 are query requests for the query key A, the query complexity of the query request 1 and the query request 2 is the same. Assuming that the correspondence between the query keyword A and the complexity value A is recorded in the first mapping table, the query complexity of the query request 1 and the query request 2 is the complexity value A for the query request 1 and the query request 2 .
  • the obtaining the correspondence between the query keyword and the complexity value may include, but is not limited to, configuring the correspondence between the query keyword and the complexity value according to experience.
  • the correspondence between the query keyword and the complexity value is trained through the neural network, and the training process is not limited.
  • the query keyword of the query request is obtained, and the complexity value of the query request is obtained. If the query is executed, the CPU resource of one core is consumed, and the memory resource of 100M is consumed.
  • the complexity value is a CPU value of one core and a complexity value corresponding to a memory resource of 100 M, and there is no limitation on this.
  • the query request may include, but is not limited to, an SQL query request; and the query keyword may include, but is not limited to, one or any combination of the following: adding a keyword (ie, a join, such as a SQL query request including a keyword join),
  • the result set is a grouped keyword (ie, groupby, such as a SQL query request including the keyword groupby), a keyword that sorts the result set (ie, orderby, such as a SQL query request including the keyword orderby), lists different keywords (ie, Distinct, such as the SQL query request includes the keyword distinct), the number of rows calculation keyword (ie count, such as SQL query request includes the keyword count), window function keyword (ie window, such as SQL query request includes the keyword window).
  • a keyword ie, a join, such as a SQL query request including a keyword join
  • the result set is a grouped keyword (ie, groupby, such as a SQL query request including the keyword groupby), a keyword that sorts the result set (ie, orderby, such
  • the complexity value 5 indicates CPU resources that consume 1 core and consumes 100 M memory resources.
  • the complexity value 10 indicates CPU resources that consume 2 cores, consumes 200 M memory resources, and so on.
  • Table 1 is only an example, and the complexity value corresponding to the query keyword is related to the actual situation, and details are not described herein again.
  • Method 1 obtaining a query keyword from the query request, and passing the query key The word queries the first mapping table to obtain a complexity value corresponding to the query keyword, and determines the complexity value as the query complexity corresponding to the query request.
  • Manner 2 Obtain a query keyword from multiple subqueries of the query request, and query the first mapping table by using each obtained query keyword to obtain a complexity value corresponding to each query keyword; then, The sum of the obtained complexity values (ie, the sum of all complexity values) is determined as the query complexity corresponding to the query request.
  • the first mapping table shown in Table 1 can be queried by querying the keyword “join” to obtain the complexity. A value of 5, then, it can be determined that the query complexity corresponding to the query request is a complexity value of 5.
  • the query request includes a subquery 1, a subquery 2, and a subquery 3.
  • the subquery 1 is a SQL join statement
  • the subquery 2 is a SQL groupby statement
  • the subquery 3 is a SQL distinct statement.
  • the sub-query 1 includes the query keyword "join”, and the first mapping table shown in Table 1 is queried by querying the keyword "join” to obtain a complexity value of 5; the sub-query 2 includes the query keyword "groupby", and the query keyword is obtained.
  • the subquery 2 includes the query keyword "distinct”, and queries the first mapping table shown in Table 1 by querying the keyword "distinct” to obtain The complexity value is 12. Then, it can be determined that the query complexity corresponding to the query request is the sum of the complexity value 5, the complexity value 10, and the complexity value 12, that is, the query complexity is the complexity value 27.
  • the amount of query data (also known as query scan data volume Query_DataScanned), that is, the amount of data returned when the query request is executed. For example, if the query request 1 is used to request data A and the size of the data A is 10M, the query data amount may be 10M, that is, the data returned to the client is 10M.
  • the historical data may be collected, and the correspondence between the data identifier and the query data amount is obtained according to the historical data; then, the correspondence between the data identifier and the query data amount is recorded in the second mapping table.
  • the front-end node may collect the above information (ie, historical data), and obtain the data A and the query data amount 100. Correspondence relationship, and record the correspondence in the second mapping table. As shown in Table 2, which is an example of the second mapping table, the content of the second mapping table is not limited.
  • the following method may be adopted: querying the second mapping table by using the data identifier of the query request, and obtaining the data identifier The corresponding amount of query data. For example, if the data identifier carried by the query request is data A, the amount of query data corresponding to the data A is determined to be 10M. If the data identifier carried in the query request is the data C, the amount of the query data corresponding to the data C is set to a default value because the second mapping table does not record the query data amount corresponding to the data C, which can be configured according to experience, such as 5M. ).
  • the query time (also known as the query time-consuming time Query_ResponseTime), that is, the time spent executing the query request (the time taken from the start of processing the query request to the end of the query request processing). For example, suppose that when query request 1 is executed, a total of 3 seconds is consumed, and the query time is 3 seconds.
  • the historical data may be collected, and the correspondence between the data identifier and the query time is obtained according to the historical data, and the correspondence between the data identifier and the query time is recorded in the second mapping table.
  • the second mapping table is queried by querying the data identifier of the request, and the query time corresponding to the data identifier is obtained.
  • the resource usage rate (also referred to as the resource usage resource_Utilization), that is, the resources consumed when the query is executed, such as the memory usage, the CPU usage, and the network bandwidth usage. It is assumed that when the query request 1 is executed, one core CPU resource, 100 M memory resource, and 100 M network bandwidth are consumed, and the resource occupancy rate is one core CPU resource, 100 M memory resource, and 100 M network bandwidth.
  • the historical data may be collected, and the correspondence between the data identifier and the resource occupancy rate may be obtained according to the historical data. Then, the correspondence between the data identifier and the resource occupancy rate may also be recorded in the second mapping table. Further, for each query request received in the preset time window, in order to obtain the resource occupancy rate of the query request, the following manner may be adopted: querying the second mapping table by using the data identifier of the query request, thereby The resource occupancy rate corresponding to the data identifier is obtained.
  • the front-end node can also maintain the second mapping table shown in Table 3.
  • the second mapping table is used to record the correspondence between the data identifier, the query data volume, the query time, and the resource occupancy rate. Based on this, for each query request received in the preset time window, the second mapping table shown in Table 3 can be identified by the data of the query request, thereby obtaining feature information corresponding to the data identifier, and the feature information is obtained. It may include one or more of query data amount, query time, and resource occupancy rate.
  • the amount of query data corresponding to the data A is 10M
  • the query time is 3 seconds
  • the resource occupancy rate is “CPU resource: 1 core; memory resource: 100M; network Bandwidth: 100M”.
  • the query data amount may be set to a default value
  • the query time is set to a default value
  • the resource is occupied.
  • the rate is set to the default value and there is no limit to this.
  • the feature information of each query request received in the preset time window can be obtained, and the feature information is an example of concurrency, query complexity, query data amount, query time, and resource occupancy rate.
  • obtaining the resource cost according to the feature information of the received query request may include: obtaining, for each query request received in the preset time window, the predicted resource amount of the query request according to the feature information of the query request. And determining the resource overhead according to the predicted resource amount of each query request, for example, the resource overhead may be the sum of the predicted resource amounts requested by each query.
  • the feature information is the query complexity
  • the greater the complexity value of the query complexity the larger the predicted resource amount, and the query complexity.
  • the smaller the complexity value the smaller the amount of predicted resources, and there is no restriction on the determination process, as long as the above rules are met.
  • the feature information is the amount of query data
  • the larger the amount of query data the larger the amount of predicted resources, the smaller the amount of query data, and the smaller the amount of predicted resources, and the determination process is not limited, as long as the above rules are met.
  • the determination process is not limited, as long as the above rule is met.
  • the feature information is the resource occupancy rate
  • the larger the resource occupancy rate is, the larger the predicted resource amount is, the smaller the resource occupancy rate is, and the smaller the predicted resource amount is.
  • the determination process is not limited, as long as the above rule is met. Of course, at least one example of the above manner is not limited thereto.
  • the five features are included as examples, and the concurrency, query complexity, and query may be used.
  • the data volume, query time, and resource occupancy rate are normalized, and the degree of concurrency, query complexity, query data volume, query time, and resource occupancy rate are normalized to the same quantity level.
  • This normalization method is not limited. Assuming that the normalized degree of concurrency A, the query complexity B, the query data amount C, the query time D, and the resource occupancy rate E, the concurrency A, the query complexity B, the query data amount C, and the query time D can be obtained.
  • the resource occupancy rate E is summed. If the summation result is larger, the larger the predicted resource amount is, the smaller the summation result is, and the smaller the prediction resource amount is. This is not limited, as long as the above rule is met.
  • weight 1* concurrency A (weight 2* query complexity B), (weight 3* query data amount C), (weight 4* query time D), (weight 5* resource occupation)
  • the rate E) is summed. If the summation result is larger, the predicted resource amount is larger, the summation result is smaller, and the predicted resource amount is smaller, and there is no limitation on this, as long as the above rule is met.
  • the weight 1, the weight 2, the weight 3, the weight 4, and the weight 5 can all be configured according to experience, and no limitation is imposed on this.
  • the sum of the weight 1, the weight 2, the weight 3, the weight 4, and the weight 5 may be 1, and may of course be other values, such as 2, 3, and the like.
  • obtaining the predicted resource amount of the query request according to the feature information of the query request may include: analyzing, by using a prediction model, the feature information of the query request to obtain a predicted resource amount of the query request; the prediction model may be Including but not limited to: Holt-Winter (three-dimensional exponential smoothing) seasonal model, ARMA (autoregressive and moving average) model, linear regression model, neural network model.
  • the prediction model may be Including but not limited to: Holt-Winter (three-dimensional exponential smoothing) seasonal model, ARMA (autoregressive and moving average) model, linear regression model, neural network model.
  • the neural network can use the historical data to train the correspondence between the feature information and the predicted resource amount.
  • the correspondence between the query complexity and the predicted resource amount can be trained.
  • the correspondence between the complexity value 5 and the predicted resource amount A can be obtained.
  • the network is a correspondence between the query complexity and the predicted resource amount through a large amount of historical data.
  • the training process is not limited. In the training result, the complexity value of the query complexity is larger, the prediction resource amount is larger, and the query complexity is greater. The smaller the complexity value, the smaller the amount of predicted resources.
  • the training process is similar for feature information such as concurrency, query data volume, query time, and resource occupancy, and will not be described here.
  • feature information such as concurrency, query data volume, query time, and resource occupancy
  • the feature information is multiple of the concurrency, the query complexity, the query data volume, the query time, and the resource occupancy rate, the training process is similar, and details are not described herein again.
  • the neural network may query the corresponding relationship according to the feature information of the query request for each query request received in the preset time window, and obtain The amount of predicted resources requested by this query is not limited to this process.
  • the above method is only an example of using the neural network model to obtain the predicted resource amount, and there is no limitation thereto.
  • the prediction model is a Holt-Winter seasonal model, an ARMA model, or a linear regression model
  • its implementation is similar to that of the neural network model, and will not be repeated here.
  • the process is consistent with the following rules:
  • the complexity value of the query complexity is larger, the amount of predicted resources is larger; when the amount of query data is larger, the amount of predicted resources is larger; when the query time is larger, the amount of predicted resources is larger.
  • the predicted resource amount is larger.
  • the computing node in the resource pool is dynamically adjusted according to the resource cost and the computing node resource, which may include, but is not limited to, obtaining the number of computing nodes according to the resource cost and the computing node resource; and then, the resource pool may be allocated and configured in the resource pool. Compute the compute nodes whose number of nodes matches.
  • the obtaining the number of the computing nodes according to the resource overhead and the computing node resources may include, but is not limited to, the rounding up the resource overhead/computing node resources, that is, the number of computing nodes may be obtained.
  • the number of the computing nodes may be obtained in other manners, as long as the number of computing nodes is greater than or equal to the resource overhead/the result of the rounding up of the computing node resources, which is not limited.
  • the computing node resources are 8 CPU cores (ie, in the resource pool).
  • Each compute node provides compute node resources for 8 CPU cores, and the number of compute nodes can be 13.
  • the number of computing nodes is 13, since 13 computing nodes can provide 104 CPU cores, 13 computing nodes can meet the resource overhead of 100 CPU cores, that is, 13 computing nodes can process the pre-processing. Set all query requests received within the time window.
  • the number of computing nodes may be 10.
  • the number of computing nodes is 10
  • 10 computing nodes can meet the resource overhead of 20G memory, that is, 10 computing nodes can process the preset time window. All query requests received.
  • the maximum number of compute nodes 13, can be determined as the number of compute nodes.
  • the computing node that matches the number of the computing nodes in the resource pool may include: if the number of the computing nodes that are already in the resource pool is smaller than the number of the computing nodes, the computing node may be expanded in the resource pool to expand the capacity. The number of subsequent compute nodes is greater than or equal to the number of compute nodes. If the number of computing nodes that are already in the resource pool is greater than the number of computing nodes, the computing node may be indented in the resource pool, so that the number of computing nodes after the shrinking is greater than or equal to the number of computing nodes.
  • the new compute nodes can be expanded in the resource pool.
  • the compute nodes are used to process all query requests received within the preset time window.
  • the computing nodes can be reduced in the resource pool, so that there are 13 computing nodes in the resource pool, and 13 The compute nodes are used to process all query requests received within the preset time window.
  • the front-end node may send a resource expansion command carrying the number of compute nodes 13 to the resource scheduling server.
  • the resource scheduling server may allocate a computing node that matches the number of computing nodes 13 in the resource pool.
  • the resource scheduling server only receives the resource expansion and contraction command carrying the number of computing nodes 13. Therefore, the computing node is expanded/reduced in the resource pool so that there are 13 computing nodes in the resource pool.
  • the resource scheduling server receives the resource expansion and contraction command carrying the number of computing nodes 13 and the resource expansion and contraction command with the number of computing nodes 8 .
  • the capacity expansion/reduction calculation in the resource pool is performed. Node so that there are 21 compute nodes in the resource pool.
  • the performance can be seconds (or even 100 milliseconds), that is, it only takes a few seconds (even to 100 milliseconds). Expand the compute node or shrink the compute node in the resource pool.
  • the querying, by the computing node in the resource pool, the data corresponding to the query request may include: for each query request received in the preset time window, the front-end node may perform SQL parsing on the query request, and utilize The SQL parsing result generates a query request, and sends the query request to the computing node; after receiving the query request, the computing node can read the data corresponding to the query request from the data source and perform calculation, and return the data to the front end node.
  • the front end node returns the received data to the client.
  • the front-end node splits the query request into six sub-query requests. There is no restriction on this process, and the six sub-query requests are load-balanced to six compute nodes.
  • the computing node For each computing node, after receiving the subquery request, the computing node reads the data corresponding to the subquery request from the data source and returns the data to the front end node. After receiving the data for the six subquery requests, the front end node combines the data together to obtain a data set, and the combined data set is the data corresponding to the query request. Then, the data set is sent to the client, and finally the data query operation is completed.
  • the resource overhead may be obtained according to the feature information of the received query request, and the number of computing nodes is obtained according to the resource cost and the computing node resource, and the allocation in the resource pool matches the number of the computing node. Compute node.
  • the computing nodes in the resource pool can be dynamically adjusted, so that the computing nodes in the resource pool can process all the received query requests, and the processing efficiency and resource utilization of the computing nodes can be more effectively improved, so that the computing nodes can be more effective.
  • Parallel processing of multiple query requests improves the utilization of CPU resources, memory resources, and network bandwidth resources, thereby achieving a better effect from the perspective of overall computing resources and user query load, and improving user experience.
  • each computing node can provide a serverless query analysis service for the user, so that the user does not need to perceive the server or the service instance, and only needs to perceive the service provided by the cloud service.
  • the user Based on the cloud service, the user only needs to input the SQL query request, and the data can be queried and analyzed by the computing node in the database, and the business analysis tools and applications can be seamlessly integrated.
  • FIG. 3 it is a schematic diagram of another application scenario of the embodiment of the present application.
  • the resource pool of the computing node can be divided into multiple sub-resource pools, with sub-resource pool 1, sub-resource pool 2, and sub-resource pool 3.
  • the compute node is located in a child resource pool.
  • the sub-resource pool 1 includes two computing nodes
  • the sub-resource pool 2 includes two computing nodes
  • the sub-resource pool 3 includes four computing nodes.
  • the computing nodes of the sub-resource pool are expanded or reduced. Processing, not for resource pools.
  • the compute node resources of all compute nodes are the same; for different sub-resource pools, the compute node resources of the compute nodes may be the same or different.
  • the computing node resource of the computing node in the sub-resource pool 1 is 4 CPU cores
  • the computing node resources of the computing node in the sub-resource pool 2 are 8 CPU cores
  • the computing node resources of the computing nodes in the sub-resource pool 3 It is 16 CPU cores.
  • the sub-resource pools of different levels may be divided into different users according to the requirements of different users.
  • the service-level agreement may be based on a contract between the network service provider and the user.
  • Information such as service type, quality of service, and customer payment is defined, and different levels of sub-resource pools are divided for different users to meet the needs of different users.
  • FIG. 4 it is a schematic flowchart of the data query method in the embodiment of the present application.
  • the method is applied to the front-end node as an example.
  • the method may include the following steps:
  • Step 401 According to the feature information of the received query request, divide the received query request into at least one allocation group; different allocation groups correspond to different sub-resource pools. For example, according to the feature information of the query request received in the preset time window, the received query request is divided into at least one allocation group.
  • Step 402 Obtain a resource overhead of the allocation group according to the feature information of the query request in the allocation group.
  • Step 403 Dynamically adjust the computing nodes in the sub-resource pool according to the resource overhead of the allocation group and the computing node resources of the sub-resource pool corresponding to the allocation group.
  • Step 404 Query, by the computing node in the sub-resource pool, data corresponding to the query request in the allocation group, that is, different query requests may be allocated to computing nodes in different sub-resource pools.
  • the above-described execution order is only an example given for convenience of description. In an actual application, the execution order between the steps may also be changed, and the order of execution is not limited. Moreover, in other embodiments, the steps of the respective methods are not necessarily performed in the order shown and described herein, and the methods may include more or less steps than those described in this specification. In addition, the individual steps described in this specification may be decomposed into a plurality of steps for description in other embodiments; the various steps described in the present specification may be combined into a single step for description in other embodiments.
  • the feature information of each query request may also be acquired first, and the feature information may include but is not limited to one or any combination of the following: concurrency, query complexity, and query data. Quantity, query time, resource occupancy. For the manner of obtaining the feature information, refer to the process shown in FIG. 2, and details are not repeatedly described herein.
  • the received query request is divided into at least one allocation group according to the feature information of the received query request, which may include, but is not limited to, a feature that can be requested according to the query for each received query request.
  • the information obtains the predicted resource amount of the query request, and determines the resource interval to which the predicted resource quantity belongs, and divides the query request into an allocation group corresponding to the resource interval; wherein different allocation groups may correspond to different resource intervals.
  • step 201 For the process of obtaining the predicted resource amount of the query request, refer to step 201, and details are not described herein again.
  • the resource interval to which the predicted resource amount belongs is determined, and the query request is allocated to the allocation group corresponding to the resource interval, which may include, but is not limited to, configuring a resource interval for each sub-resource pool in advance, and the configuration manner is not limited. For example, when the computing node resource of the sub-resource pool is larger, the resource interval of the sub-resource pool may be larger. When the computing node resource of the sub-resource pool is smaller, the resource interval of the sub-resource pool may be smaller.
  • the compute node resource of the sub-resource pool 1 is 4 CPU cores
  • the compute node resource of the sub-resource pool 2 is 8 CPU cores
  • the compute node resources of the sub-resource pool 3 are 16 CPU cores
  • the sub-resource pool 1 The resource interval is [0-1) CPU cores
  • the resource interval of sub-resource pool 2 is [1-2) CPU cores
  • the resource interval of sub-resource pool 3 is [2-infinity" CPU cores.
  • an allocation group may be configured for each resource interval, for example, allocation group 1 is configured for the resource interval of the sub-resource pool 1, and allocation group 2 is configured for the resource interval of the sub-resource pool 2, and the resource interval configuration for the sub-resource pool 3 is configured. Assign group 3.
  • allocation group 1 corresponds to sub-resource pool 1
  • allocation group 2 corresponds to sub-resource pool 2
  • allocation group 3 corresponds to sub-resource pool 3.
  • the resource interval to which the predicted resource amount belongs may be determined as the resource interval of the sub-resource pool 2, and the query request may be divided into the allocation group 2.
  • the query requests can be divided into the respective allocation groups, for example, the query request 1-10 is divided into the allocation group 1, and the query request is 11-50. Divided into allocation group 2, query requests 51-100 are divided into allocation group 3.
  • obtaining the resource cost of the allocation group according to the feature information of the query request in the allocation group may include: obtaining, for each query request in the allocation group, the predicted resource of the query request according to the feature information of the query request. And obtain the resource overhead of the allocation group according to the predicted resource amount.
  • step 402 For the implementation process of step 402, refer to step 201. The difference is that in step 201, all the query requests are processed, and in step 402, all the query requests in the allocation group are processed. The other processes are similar, and the details are not repeated here.
  • dynamically adjusting the computing node in the sub-resource pool according to the resource cost of the allocation group and the computing node resource of the sub-resource pool corresponding to the allocation group may include: according to the resource overhead of the allocation group and the allocation group
  • the computing node resource of the corresponding sub-resource pool obtains the number of computing nodes in the sub-resource pool; and allocates a computing node that matches the number of the computing nodes in the sub-resource pool.
  • allocating the computing node that matches the number of the computing nodes in the sub-resource pool may include: if the number of computing nodes that already exist in the sub-resource pool is less than the number of computing nodes, expanding the sub-resource pool Computational node, the number of computing nodes after expansion is greater than or equal to the number of computing nodes; if the number of computing nodes already existing in the sub-resource pool is greater than the number of computing nodes, the computing node is reduced in the sub-resource pool, and the capacity is reduced.
  • the number of compute nodes is greater than or equal to the number of compute nodes.
  • step 403 can be referred to step 202.
  • the computing node in the resource pool is dynamically adjusted according to the resource overhead and computing node resources of all the received query requests
  • step 403 The computing node in the sub-resource pool is dynamically adjusted according to the resource overhead of the allocation group and the computing node resources of the sub-resource pool corresponding to the allocation group.
  • the number of computing nodes 10 in the sub-resource pool 1 can be obtained according to the resource overhead of the allocation group 1 and the computing node resources of the sub-resource pool 1, and 10 computing nodes are allocated in the sub-resource pool 1.
  • the number of computing nodes 8 in the sub-resource pool 2 can be obtained according to the resource overhead of the allocation group 2 and the computing node resources of the sub-resource pool 2, and 8 computing nodes are allocated in the sub-resource pool 1.
  • the number of computing nodes 13 in the sub-resource pool 3 can be obtained according to the resource overhead of the allocation group 3 and the computing node resources of the sub-resource pool 3, and 13 computing nodes are allocated in the sub-resource pool 3.
  • step 404 refers to the step 203.
  • the query request corresponding to the query request is sent to the computing node of the resource pool.
  • the query request of the group 1 is allocated.
  • the corresponding query request is sent to the computing node of the child resource pool 1
  • the query request corresponding to the query request of the group 2 is sent to the computing node of the child resource pool 2
  • the query request corresponding to the query request of the group 3 is sent to the child resource.
  • the compute nodes of pool 3 are not repeated here.
  • the embodiment of the present application further provides a data query device, as shown in FIG. 5, which is a structural diagram of the device, and the device includes:
  • the obtaining module 501 is configured to obtain a resource overhead according to the feature information of the received query request; the processing module 502 dynamically adjusts the computing node in the resource pool according to the resource cost and the computing node resource; and the query module 503 is configured to pass the computing node. Query data corresponding to the query request.
  • the obtaining module 501 is further configured to: when the feature information includes the query complexity, obtain a query keyword from the query request; query the first mapping table by using the query keyword, and obtain the key with the query
  • the complexity value corresponding to the word the complexity value is determined as the query complexity corresponding to the query request; or the query keyword is obtained from multiple subqueries of the query request; and the first map is queried by the obtained query keyword a table, the complexity value corresponding to the query keyword is obtained; the sum of the obtained complexity values is determined as a query complexity corresponding to the query request; wherein the first mapping table is used to record query keywords and complexity The correspondence of values.
  • the embodiment of the present application provides a data query device, including a processor, configured to obtain a resource cost according to the feature information of the received query request, and dynamically adjust the resource according to the resource cost and the computing node resource.
  • a computing node in the pool querying, by the computing node, data corresponding to the query request.
  • the embodiment of the present application further provides a machine readable storage medium, which can be applied to a data query device, where a plurality of computer instructions are stored on a machine readable storage medium; wherein the computer instructions are executed And performing the following processing: obtaining resource overhead according to the feature information of the received query request; dynamically adjusting the computing node in the resource pool according to the resource overhead and the computing node resource; and querying, by the computing node, data corresponding to the query request .
  • the embodiment of the present application further provides a data query device, as shown in FIG. 6, which is a structural diagram of the device, and the device includes:
  • the dividing module 601 is configured to divide the received query request into at least one allocation group according to the feature information of the received query request; the different allocation groups correspond to different sub-resource pools; and the obtaining module 602 is configured to perform the query according to the allocation group
  • the requested feature information obtains the resource cost of the allocation group;
  • the processing module 603 is configured to dynamically adjust the resource pool in the sub-resource pool according to the resource cost of the allocation group and the computing node resource of the sub-resource pool corresponding to the allocation group.
  • the computing node is configured to query, by the computing node in the sub-resource pool, data corresponding to the query request in the allocation group.
  • the dividing module 603 is specifically configured to: obtain, according to the received query request, the predicted resource quantity of the query request according to the feature information of the query request, and determine a resource interval to which the predicted resource quantity belongs;
  • the query request is divided into allocation groups corresponding to the resource intervals; wherein different allocation groups correspond to different resource intervals.
  • the embodiment of the present application provides a data query device, including a processor, configured to divide a received query request into at least one allocation group according to the feature information of the received query request, where
  • the different allocation groups correspond to different sub-resource pools; the resource overhead of the allocation group is obtained according to the feature information of the query request in the allocation group; and the resource cost of the allocation group and the sub-resource pool corresponding to the allocation group are calculated according to the resource cost of the allocation group
  • the node resource dynamically adjusts the computing node in the sub-resource pool; and the computing node in the sub-resource pool queries the data corresponding to the query request in the allocation group.
  • the embodiment of the present application further provides a machine readable storage medium, which can be applied to a data query device, where a plurality of computer instructions are stored on a machine readable storage medium; wherein the computer instructions are executed
  • the processing is as follows: according to the feature information of the received query request, the received query request is divided into at least one allocation group; wherein different allocation groups correspond to different sub-resource pools; according to the characteristics of the query request in the allocation group
  • the information obtains the resource overhead of the allocation group; dynamically adjusts the computing node in the sub-resource pool according to the resource overhead of the allocation group and the computing node resource of the sub-resource pool corresponding to the allocation group;
  • the computing node queries data corresponding to the query request in the allocation group.
  • the system, device, module or unit illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product having a certain function.
  • a typical implementation device is a computer, and the specific form of the computer may be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, and a game control.
  • embodiments of the present application can be provided as a method, system, or computer program product.
  • the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware.
  • embodiments of the present application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • these computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the instruction means implements the functions specified in one or more blocks of the flowchart or in a flow or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Mathematical Physics (AREA)
  • Fuzzy Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请提供一种数据查询方法、装置及设备，该方法包括：根据接收到的查询请求的特征信息获得资源开销；根据所述资源开销和计算节点资源动态调节资源池中的计算节点；通过所述计算节点查询与所述查询请求对应的数据。通过本申请的技术方案，可以动态调节资源池中的计算节点，使得资源池中的计算节点能够处理接收到的所有查询请求，更有效的提高计算节点的处理效率和资源利用率，使得计算节点能够更有效的并行处理多个查询请求，提高CPU资源、内存资源、网络带宽资源的利用率，从而从整体计算资源和用户查询负载角度达到一个更好的效果，提高用户使用感受。

Description

一种数据查询方法、装置及设备
本申请要求2018年03月29日递交的申请号为201810268968.0、发明名称为“一种数据查询方法、装置及设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及互联网技术领域，尤其涉及一种数据查询方法、装置及设备。
背景技术
开放分析服务(Open Analytics)用于为用户提供无服务器化(Serverless)的查询分析服务，能够对海量数据进行任意维度的分析和查询，支持高并发、低延时(毫秒级响应)、实时在线分析、海量数据查询等功能。开放分析服务系统中，可以包括数据源和计算节点，数据源用于存储大量数据，计算节点在接收到查询请求后，从数据源中查询与该查询请求对应的数据。
但是，在某些应用场景下(如地图数据的查询场景、画像数据的查询场景等)，计算节点可能在短时间内接收到多个查询请求(即并发数很高)，即需要在短时间内处理多个查询请求，导致CPU(Central Processing Unit，中央处理器)资源、内存资源、网络带宽等出现异常，从而导致查询超时或者查询失败。
发明内容
本申请提供一种数据查询方法，所述方法包括：
根据接收到的查询请求的特征信息获得资源开销；
根据所述资源开销和计算节点资源动态调节资源池中的计算节点；
通过所述计算节点查询与所述查询请求对应的数据。
本申请提供一种数据查询方法，所述方法包括：
根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；其中，不同的分配组对应不同的子资源池；
根据分配组中的查询请求的特征信息获得所述分配组的资源开销；
根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点；
通过子资源池中的计算节点查询与所述分配组中的查询请求对应的数据。
本申请提供一种一种数据查询装置，所述装置包括：
获得模块，用于根据接收到的查询请求的特征信息获得资源开销；
处理模块，根据资源开销和计算节点资源动态调节资源池中的计算节点；
查询模块，用于通过所述计算节点查询与所述查询请求对应的数据。
本申请提供一种数据查询装置，所述装置包括：
划分模块，用于根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；其中，不同分配组对应不同的子资源池；
获得模块，用于根据分配组中的查询请求的特征信息获得所述分配组的资源开销；
处理模块，用于根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点；
查询模块，用于通过所述子资源池中的计算节点查询与所述分配组中的查询请求对应的数据。
本申请提供一种数据查询设备，包括：处理器，用于根据接收到的查询请求的特征信息获得资源开销；根据资源开销和计算节点资源动态调节资源池中的计算节点；通过所述计算节点查询与所述查询请求对应的数据。
本申请提供一种数据查询设备，包括：处理器，用于根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；其中，不同的分配组对应不同的子资源池；根据分配组中的查询请求的特征信息获得所述分配组的资源开销；根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点；通过子资源池中的计算节点查询与所述分配组中的查询请求对应的数据。
基于上述技术方案，本申请实施例中，可以根据接收到的查询请求的特征信息获得资源开销，并根据资源开销和计算节点资源动态调节资源池中的计算节点，使得资源池中的计算节点能够处理接收到的所有查询请求，更有效的提高计算节点的处理效率和资源利用率，可以使得计算节点能够更有效的并行处理多个查询请求，提高CPU资源、内存资源、网络带宽资源的利用率，从而从整体计算资源和用户查询负载角度达到一个更好的效果，提高用户使用感受。而且，通过动态调节资源池中的计算节点，使得各计算节点可以为用户提供无服务器化(Serverless)的查询分析服务，使得用户无需感知服务器或者服务实例，只需感知云服务提供的服务本身，基于云服务，用户只需要输入SQL查询请求，就可以由计算节点在数据源中进行数据查询和分析，可以无缝集成商业分析 工具和应用程序。可以对资源进行智能分析和自动调整，更有效的提高云数据库和云数据分析服务集群的资源利用率和性价比。
附图说明
为了更加清楚地说明本申请实施例或者现有技术中的技术方案，下面将对本申请实施例或者现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请中记载的一些实施例，对于本领域普通技术人员来讲，还可以根据本申请实施例的这些附图获得其它的附图。
图1是本申请一种实施方式中的系统结构示意图；
图2是本申请一种实施方式中的数据查询方法的流程图；
图3是本申请另一种实施方式中的系统结构示意图；
图4是本申请另一种实施方式中的数据查询方法的流程图；
图5是本申请一种实施方式中的数据查询装置的结构图；
图6是本申请另一种实施方式中的数据查询装置的结构图。
具体实施方式
在本申请实施例使用的术语仅仅是出于描述特定实施例的目的，而非限制本申请。本申请和权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其它含义。还应当理解，本文中使用的术语“和/或”是指包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本申请实施例可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本申请范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，此外，所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
本申请实施例提出一种数据查询方法，该方法可以应用于包括客户端、负载均衡设备、前端节点(front node，也可以称为前端服务器)、计算节点(compute node，也可以称为计算服务器)和数据源的系统，如用于实现开放分析服务的系统。当然，还可以包括其它服务器，如资源调度服务器等，对此不做限制。
参见图1所示，为本申请实施例的应用场景示意图，在前端节点的资源池中，包括 一个或多个前端节点，图1以3个前端节点为例。在计算节点的资源池中，包括一个或多个计算节点，图1以5个计算节点为例。实际应用中，可以对前端节点进行扩容(增加前端节点的数量)或者缩容(减少前端节点的数量)，也可以对计算节点进行扩容(增加计算节点的数量)或者缩容(减少计算节点的数量)，本实施例正是针对计算节点进行扩容或者缩容的方案。
其中，客户端用于从数据源中查询数据，如可以是终端设备(如PC(Personal Computer，个人计算机)、笔记本电脑、移动终端等)包括的APP(Application，应用)，也可以是终端设备包括的浏览器，对此客户端的类型不做具体限制。
其中，负载均衡设备用于对查询请求进行负载均衡，例如，在接收到大量查询请求后，可以将这些查询请求负载均衡到各前端节点，对此过程不做限制。
其中，数据源用于存储各种类型的数据，且能够将数据源中存储的数据提供给客户端。对于数据源中存储的数据的类型，本申请实施例中不做限制，如可以是用户数据、商品数据、地图数据、视频数据、图像数据、音频数据等。
其中，资源池中的多个前端节点用于提供相同的功能。具体的，前端节点用于接收客户端发送的查询请求，并对所述查询请求进行SQL(Structured Query Language，结构化查询语言)解析，利用SQL解析结果生成查询请求，并将该查询请求发送给计算节点，该查询请求用于请求与该查询请求对应的数据。然后，前端节点还用于接收计算节点返回的数据，并将该数据发送给客户端。
其中，资源池中的多个计算节点用于提供相同的功能。具体的，计算节点用于接收前端节点发送的查询请求，并利用该查询请求从数据源中读取与该查询请求对应的数据，对此读取过程不做限制，并将该数据发送给前端节点。
在一个例子中，若计算节点在短时间内接收到大量查询请求(即并发数很高)，则计算节点需要在短时间内处理大量查询请求，导致CPU资源、内存资源、网络带宽等出现异常，从而导致查询超时或者查询失败。与上述方式不同的是，本申请实施例中，可以动态调节资源池中的计算节点，即当存在大量查询请求时，可以增加资源池中的计算节点数量，减少每个计算节点的查询请求数量，从而避免某个计算节点在短时间内处理大量查询请求，更有效的提高计算节点的处理效率和资源利用率，减轻CPU资源、内存资源、网络带宽的占用，可以提高处理性能，并避免客户端查询超时或者失败，提高用户使用感受。
在上述应用场景下，参见图2所示，为本申请实施例中提出的数据查询方法的流程 示意图，该方法可以应用于数据查询设备，该数据查询设备可以为图1中的负载均衡设备、或者前端节点、或者资源调度服务器，对此不做限制，本实施例中以应用于前端节点为例，该方法可以包括以下步骤：
步骤201，根据接收到的查询请求的特征信息获得资源开销。例如，可以根据预设时间窗内接收到的查询请求的特征信息获得资源开销。
步骤202，根据该资源开销和计算节点资源动态调节资源池中的计算节点。
步骤203，通过资源池中的计算节点查询与该查询请求对应的数据。
在一个例子中，上述执行顺序只是为了方便描述给出的一个示例，在实际应用中，还可以改变步骤之间的执行顺序，对此执行顺序不做限制。而且，在其它实施例中，并不一定按照本说明书示出和描述的顺序来执行相应方法的步骤，其方法所包括的步骤可以比本说明书所描述的更多或更少。此外，本说明书中所描述的单个步骤，在其它实施例中可能被分解为多个步骤进行描述；本说明书中所描述的多个步骤，在其它实施例也可能被合并为单个步骤进行描述。
其中，当客户端需要请求数据源中的数据时，可以发送查询请求，负载均衡设备在接收到该查询请求后，可以将该查询请求发送给前端节点，前端节点在接收到该查询请求后，可以将该查询请求存储到查询队列(Query Queue)中。
其中，前端节点可以设置预设时间窗，该预设时间窗的时间可以根据经验配置，如3秒等。基于此，前端节点可以将预设时间窗内存储在查询队列中的所有查询请求，确定为预设时间窗内接收到的查询请求，如100个查询请求。
在执行步骤201之前，针对预设时间窗内接收到的所有查询请求，还可以先获取每个查询请求的特征信息，该特征信息可以包括但不限于以下之一或者任意组合：并发度、查询复杂度、查询数据量、查询时间、资源占用率。
一、并发度(Concurrency)，即预设时间窗内接收的查询请求数量，如100。
二、查询复杂度(Query_Complexity)，即执行查询请求的复杂程度，可以表示查询时间、CPU资源占用、内存资源占用、网络带宽占用等情况。其中，查询复杂度通常为一个数值，可以通过对查询时间、CPU资源占用、内存资源占用、网络带宽占用进行归一化，得到查询复杂度的数值。例如，若执行查询请求1时，需要占用大量CPU资源、内存资源、网络带宽，查询时间较长，则查询请求1的查询复杂度较高。若执行查询请求2时，需要占用少量CPU资源、内存资源、网络带宽，查询时间较短，则查询请求2的查询复杂度较低。
针对具有相同查询关键字的查询请求，其查询复杂度相同或者类似，因此，可以获取查询关键字与复杂度值的对应关系，并在第一映射表中记录查询关键字与复杂度值的对应关系。例如，假设查询请求1和查询请求2均是针对查询关键字A的查询请求，则查询请求1和查询请求2的查询复杂度相同。假设在第一映射表中记录查询关键字A与复杂度值A的对应关系，则对于查询请求1和查询请求2来说，查询请求1和查询请求2的查询复杂度均是复杂度值A。
其中，获取查询关键字与复杂度值的对应关系，可以包括但不限于：根据经验配置查询关键字与复杂度值的对应关系。或者，通过神经网络训练查询关键字与复杂度值的对应关系，对此训练过程不做限制。或者，在执行某查询请求时，获取该查询请求的查询关键字，并获取该查询请求的复杂度值，如执行该查询请求时，消耗1个核的CPU资源、消耗100M的内存资源，则复杂度值是1个核的CPU资源、100M的内存资源对应的复杂度值，对此不做限制。
在一个例子中，查询请求可以包括但不限于SQL查询请求；且查询关键字可以包括但不限于以下之一或者任意组合：加入关键字(即join，如SQL查询请求包括关键字join)、对结果集进行分组的关键字(即groupby，如SQL查询请求包括关键字groupby)、对结果集进行排序的关键字(即orderby，如SQL查询请求包括关键字orderby)、列出不同关键字(即distinct，如SQL查询请求包括关键字distinct)、行数计算关键字(即count，如SQL查询请求包括关键字count)、窗口函数关键字(即window，如SQL查询请求包括关键字window)。
参见表1所示，为第一映射表的一个示例，其用于记录查询关键字与复杂度值的对应关系，这里的复杂度值体现了查询请求的复杂度。例如，复杂度值5表示消耗1个核的CPU资源、消耗100M的内存资源，复杂度值10表示消耗2个核的CPU资源、消耗200M的内存资源，以此类推。当然，表1只是一个示例，对于查询关键字对应的复杂度值，与实际情况有关，在此不再赘述。
表1
查询关键字 复杂度值
join 复杂度值5
groupby 复杂度值10
orderby 复杂度值8
distinct 复杂度值12
count 复杂度值6
window 复杂度值15
进一步的，针对预设时间窗内接收到的每个查询请求，为了获取该查询请求的查询复杂度，可以采用如下方式：方式一、从该查询请求中获取查询关键字，并通过该查询关键字查询第一映射表，以得到与该查询关键字对应的复杂度值，并将该复杂度值确定为该查询请求对应的查询复杂度。方式二、从该查询请求的多个子查询中获取查询关键字，并通过获取的每个查询关键字查询第一映射表，以得到与每个查询关键字对应的复杂度值；然后，可以将得到的复杂度值之和(即所有复杂度值的和)确定为该查询请求对应的查询复杂度。
例如，针对方式一，假设查询请求是SQL的join语句，即该查询请求包括查询关键字“join”，则可以通过查询关键字“join”查询表1所示的第一映射表，得到复杂度值5，然后，可以确定该查询请求对应的查询复杂度为复杂度值5。
针对方式二，假设查询请求包括子查询1、子查询2、子查询3，该子查询1是SQL的join语句，该子查询2是SQL的groupby语句，该子查询3是SQL的distinct语句。子查询1包括查询关键字“join”，通过查询关键字“join”查询表1所示的第一映射表，得到复杂度值5；子查询2包括查询关键字“groupby”，通过查询关键字“groupby”查询表1所示的第一映射表，得到复杂度值10；子查询2包括查询关键字“distinct”，通过查询关键字“distinct”查询表1所示的第一映射表，得到复杂度值12。然后，可以确定该查询请求对应的查询复杂度为复杂度值5、复杂度值10与复杂度值12之间的和，即查询复杂度为复杂度值27。
三、查询数据量(也可以称为查询扫描数据量Query_DataScanned)，即执行查询请求时返回的数据量。例如，假设查询请求1用于请求数据A，而数据A的大小为10M，则查询数据量可以是10M，即向客户端返回的数据是10M。
在一个例子中，可以收集历史数据，并根据历史数据获取数据标识与查询数据量的对应关系；然后，在第二映射表中记录该数据标识与该查询数据量的对应关系。例如，在执行某查询请求时，若该查询请求用于请求数据A，而数据A的大小为10M，则前端节点可以收集到上述信息(即历史数据)，并获取数据A与查询数据量100的对应关系，并在第二映射表中记录该对应关系。参见表2所示，为第二映射表的一个示例，对此第二映射表的内容不做限制。
表2
数据标识 查询数据量
数据A 10M
数据B 20M
进一步的，针对预设时间窗内接收到的每个查询请求，为了获取该查询请求的查询数据量，可以采用如下方式：通过该查询请求的数据标识查询第二映射表，得到与该数据标识对应的查询数据量。例如，若该查询请求携带的数据标识为数据A，则确定与数据A对应的查询数据量10M。若该查询请求携带的数据标识为数据C，由于第二映射表未记录数据C对应的查询数据量10M，则可以将数据C对应的查询数据量设置为默认值(可以根据经验配置，如5M)。
四、查询时间(也可以称为查询耗时时间Query_ResponseTime)，即执行查询请求时所消耗的时间(从开始处理查询请求到查询请求处理结束所消耗的时间)。例如，假设执行查询请求1时，共消耗3秒，则查询时间是3秒。
其中，可以收集历史数据，根据历史数据获取数据标识与查询时间的对应关系，在第二映射表中记录该数据标识与查询时间的对应关系。针对预设时间窗内接收到的每个查询请求，为了获取该查询请求的查询时间，采用如下方式：通过查询请求的数据标识查询第二映射表，得到与该数据标识对应的查询时间。
五、资源占用率(也称为资源利用率Resource_Utilization)，即执行查询请求时所消耗的资源，如内存占用率、CPU占用率、网络带宽占用率等。假设执行查询请求1时，消耗1个核的CPU资源、100M的内存资源、100M的网络带宽，则资源占用率为1个核的CPU资源、100M的内存资源、100M的网络带宽。
其中，可以通过收集历史数据，并根据所述历史数据获取数据标识与资源占用率的对应关系。然后，还可以在第二映射表中记录该数据标识与该资源占用率的对应关系。进一步的，针对预设时间窗内接收到的每个查询请求，为了获取该查询请求的资源占用率，还可以采用如下方式：通过该查询请求的数据标识查询所述第二映射表，从而可以得到与该数据标识对应的资源占用率。
在一个例子中，前端节点还可以维护表3所示的第二映射表，该第二映射表用于记录数据标识、查询数据量、查询时间、资源占用率的对应关系。基于此，针对预设时间窗内接收到的每个查询请求，可以通过该查询请求的数据标识查询表3所示的第二映射表，从而得到与该数据标识对应的特征信息，该特征信息可以包括查询数据量、查询时 间、资源占用率中的一个或者多个。
表3
Figure PCTCN2019078418-appb-000001
综上所述，若上述查询请求携带的数据标识为数据A，则确定与数据A对应的查询数据量10M、查询时间3秒，资源占用率“CPU资源：1核；内存资源：100M；网络带宽：100M”。此外，若查询请求携带的数据标识为数据C，由于第二映射表未记录数据C对应的内容，则可以将查询数据量设置为默认值、并将查询时间设置为默认值、并将资源占用率设置为默认值，对此不做限制。
经过上述过程，可以得到预设时间窗内接收到的每个查询请求的特征信息，以特征信息为并发度、查询复杂度、查询数据量、查询时间、资源占用率为例。
在步骤201中，根据接收到的查询请求的特征信息获得资源开销，可以包括：针对预设时间窗内接收到的每个查询请求，根据该查询请求的特征信息获得该查询请求的预测资源量，并根据每个查询请求的预测资源量确定资源开销，例如，资源开销可以为每个查询请求的预测资源量之和。
其中，在根据该查询请求的特征信息获得该查询请求的预测资源量时，假设该特征信息为查询复杂度，则查询复杂度的复杂度值越大时，预测资源量越大，查询复杂度的复杂度值越小时，预测资源量越小，对此确定过程不做限制，只要符合上述规律即可。假设该特征信息为查询数据量，则查询数据量越大时，预测资源量越大，查询数据量越小时，预测资源量越小，对此确定过程不做限制，只要符合上述规律即可。假设该特征信息为查询时间，则查询时间越大时，预测资源量越大，查询时间越小时，预测资源量越小，对此确定过程不做限制，只要符合上述规律即可。假设该特征信息为资源占用率，则资源占用率越大时，预测资源量越大，资源占用率越小时，预测资源量越小，对此确定过程不做限制，只要符合上述规律即可。当然，上述方式至少一个示例，对此不做限制。
例如，当该特征信息为并发度、查询复杂度、查询数据量、查询时间、资源占用率中的多个时，以包括这5个特征为例，则可以对并发度、查询复杂度、查询数据量、查询时间、资源占用率进行归一化，即将并发度、查询复杂度、查询数据量、查询时间、 资源占用率归一化到同一数量级别，对此归一化方式不做限制。假设得到归一化后的并发度A、查询复杂度B、查询数据量C、查询时间D、资源占用率E，则可以对并发度A、查询复杂度B、查询数据量C、查询时间D、资源占用率E进行求和，若求和结果越大时，预测资源量越大，求和结果越小时，预测资源量越小，对此不做限制，只要符合上述规律即可。
又例如，还可以对(权重1*并发度A)、(权重2*查询复杂度B)、(权重3*查询数据量C)、(权重4*查询时间D)、(权重5*资源占用率E)进行求和，若求和结果越大时，预测资源量越大，求和结果越小时，预测资源量越小，对此不做限制，只要符合上述规律即可。其中，权重1、权重2、权重3、权重4和权重5均可以根据经验配置，对此不做限制。例如，权重1、权重2、权重3、权重4、权重5的和可以为1，当然也可以为其它数值，如2、3等。
在一个例子中，根据该查询请求的特征信息获得该查询请求的预测资源量，可以包括：通过预测模型对该查询请求的特征信息进行分析，得到该查询请求的预测资源量；该预测模型可以包括但不限于：Holt-Winter(三次指数平滑法)季节模型、ARMA(自回归与滑动平均)模型、线性回归模型、神经网络模型。
以预测模型是神经网络模型为例，则神经网络可以利用历史数据训练特征信息与预测资源量的对应关系。例如，特征信息是查询复杂度时，则可以训练查询复杂度与预测资源量的对应关系。例如，在执行某个查询请求时，假设其查询复杂度为复杂度值5，实际消耗的资源量为资源量A，则可以得到复杂度值5与预测资源量A的对应关系，当然，神经网络是通过大量历史数据训练查询复杂度与预测资源量的对应关系，对此训练过程不做限制，在训练结果中，查询复杂度的复杂度值越大，预测资源量越大，查询复杂度的复杂度值越小，预测资源量越小。对于并发度、查询数据量、查询时间、资源占用率等特征信息，其训练过程类似，在此不再赘述。当该特征信息为并发度、查询复杂度、查询数据量、查询时间、资源占用率中的多个时，其训练过程类似，在此不再赘述。
进一步的，在神经网络训练出特征信息与预测资源量的对应关系后，针对预设时间窗内接收到的每个查询请求，神经网络可以根据该查询请求的特征信息查询所述对应关系，获得该查询请求的预测资源量，对此过程不做限制。
当然，上述方式只是利用神经网络模型得到预测资源量的一个示例，对此不做限制。当预测模型是Holt-Winter季节模型、ARMA模型、线性回归模型时，其实现方式与神经网络模型的实现方式类似，在此不再重复赘述。总之，只要确定过程符合以下规律即可： 查询复杂度的复杂度值越大时，预测资源量越大；查询数据量越大时，预测资源量越大；查询时间越大时，预测资源量越大；资源占用率越大时，预测资源量越大；并发数越大时，预测资源量越大。
在步骤202中，根据资源开销和计算节点资源动态调节资源池中的计算节点，可以包括但不限于：根据资源开销和计算节点资源获得计算节点数量；然后，可以在资源池中分配与所述计算节点数量匹配的计算节点。
其中，根据该资源开销和计算节点资源获得计算节点数量，可以包括但不限于如下方式：对该资源开销/计算节点资源向上取整，即可以得到计算节点数量。当然，还可以采用其它方式获得计算节点数量，只要计算节点数量大于等于资源开销/计算节点资源向上取整的结果即可，对此不做限制。
例如，当预设时间窗内接收到的所有查询请求的预测资源量之和为100个CPU核，即资源开销是100个CPU核时，假设计算节点资源为8个CPU核(即资源池中的每个计算节点均提供8个CPU核的计算节点资源)，则计算节点数量可以为13个。显然，当计算节点数量为13个时，由于13个计算节点可以提供104个CPU核，因此，13个计算节点能够满足100个CPU核的资源开销，也就是说，13个计算节点能够处理预设时间窗内接收到的所有查询请求。
又例如，当资源开销是20G内存时，假设计算节点资源为2G内存，则计算节点数量可以为10个。显然，当计算节点数量为10个时，由于10个计算节点可以提供20G内存，因此，10个计算节点能够满足20G内存的资源开销，也就是说，10个计算节点能够处理预设时间窗内接收到的所有查询请求。
又例如，当资源开销是100个CPU核、20G内存，计算节点资源为8个CPU核、2G内存时，则CPU核资源需要使用13个计算节点，内存资源需要使用10个计算节点，因此，可以将最大的计算节点数量13，确定为计算节点数量。
其中，在资源池中分配与该计算节点数量匹配的计算节点，可以包括：若资源池中已经存在的计算节点的数量小于该计算节点数量，则可以在资源池中扩容计算节点，以使扩容后的计算节点的数量大于等于该计算节点数量。若资源池中已经存在的计算节点的数量大于该计算节点数量，则可以在资源池中缩容计算节点，以使缩容后的计算节点的数量大于等于该计算节点数量。
例如，假设资源池中已经存在8个计算节点，而上述计算节点数量为13个，则可以在资源池中新扩容5个计算节点，这样，资源池中一共存在13个计算节点，而这13个 计算节点用于处理预设时间窗内接收到的所有查询请求。
又例如，假设资源池中已经存在20个计算节点，而上述计算节点数量为13个，则可以在资源池中缩容7计算节点，这样，资源池中一共存在13个计算节点，而这13个计算节点用于处理预设时间窗内接收到的所有查询请求。
在一个例子中，前端节点在获得计算节点数量13后，可以向资源调度服务器发送携带计算节点数量13的资源扩缩容命令。资源调度服务器在接收到该资源扩缩容命令后，就可以在资源池中分配与该计算节点数量13匹配的计算节点。
例如，若存在一个前端节点，则资源调度服务器只接收到携带计算节点数量13的资源扩缩容命令，因此，在资源池中扩容/缩容计算节点，以使资源池中存在13个计算节点。又例如，若存在两个前端节点，假设资源调度服务器接收到携带计算节点数量13的资源扩缩容命令、携带计算节点数量8的资源扩缩容命令，则在资源池中扩容/缩容计算节点，以使资源池中存在21个计算节点。
其中，资源调度服务器在资源池中扩容/缩容计算节点时，性能可以是秒级(甚至能优化到百毫秒级)，即只需要数秒钟时间(甚至能优化到百毫秒级)，就可以在资源池中扩容计算节点或者缩容计算节点。
在步骤203中，通过资源池中的计算节点查询与上述查询请求对应的数据，可以包括：针对预设时间窗内接收到的每个查询请求，前端节点可以对该查询请求进行SQL解析，利用SQL解析结果生成查询请求，并将该查询请求发送给计算节点；计算节点在接收到查询请求后，可以从数据源读取与该查询请求对应的数据并进行计算，并将数据返回给前端节点；前端节点将接收到的数据返回给客户端。例如，前端节点将查询请求拆分成6个子查询请求，对此过程不做限制，并将6个子查询请求负载均衡到6个计算节点。对于每个计算节点来说，计算节点接收到子查询请求后，从数据源读取与该子查询请求对应的数据，并将数据返回给前端节点。前端节点在接收到针对6个子查询请求的数据后，将这些数据组合在一起，得到数据集合，而组合后的数据集合就是上述查询请求对应的数据。然后，将该数据集合发送给客户端，最终完成数据查询操作。
基于上述技术方案，本申请实施例中，可以根据接收到的查询请求的特征信息获得资源开销，并根据资源开销和计算节点资源获得计算节点数量，并在资源池中分配与该计算节点数量匹配的计算节点。这样，可以动态调节资源池中的计算节点，使得资源池中的计算节点能够处理接收到的所有查询请求，更有效的提高计算节点的处理效率和资源利用率，可以使得计算节点能够更有效的并行处理多个查询请求，提高CPU资源、内 存资源、网络带宽资源的利用率，从而从整体计算资源和用户查询负载角度达到一个更好的效果，提高用户使用感受。通过对查询请求的特征进行分析和预测，可以对计算节点的资源进行智能分析和自动调整，更有效的提高云数据库和云数据分析服务集群的资源利用率和性价比。而且，通过动态调节资源池中的计算节点，使得各计算节点可以为用户提供无服务器化(Serverless)的查询分析服务，使得用户无需感知服务器或者服务实例，只需感知云服务提供的服务本身，基于云服务，用户只需要输入SQL查询请求，就可以由计算节点在数据库中进行数据查询和分析，可以无缝集成商业分析工具和应用程序。
参见图3所示，为本申请实施例的另一应用场景示意图，以下对图3和图1的不同之处进行说明。在图1中，所有计算节点都位于同一个资源池，在图3中，可以将计算节点的资源池划分为多个子资源池，以子资源池1、子资源池2、子资源池3为例，而计算节点是位于子资源池。例如，子资源池1包括2个计算节点，子资源池2包括2个计算节点，子资源池3包括4个计算节点，本实施例中，是对子资源池的计算节点进行扩容或者缩容处理，而不是针对资源池。
针对同一个子资源池，所有计算节点的计算节点资源相同；对于不同的子资源池，计算节点的计算节点资源可以相同或者不同。例如，子资源池1内的计算节点的计算节点资源为4个CPU核，子资源池2内的计算节点的计算节点资源为8个CPU核，子资源池3内的计算节点的计算节点资源为16个CPU核。
其中，可以根据不同用户的需求，为不同用户划分不同级别的子资源池，例如，可以基于用户的SLA(Service-Level Agreement，服务等级协议，即网络服务供应商和用户间的一份合同，定义了服务类型、服务质量和客户付款等术语)信息，为不同用户划分不同级别的子资源池，从而满足不同用户的需求。
在上述应用场景下，参见图4所示，为本申请实施例中提出的数据查询方法的流程示意图，以该方法应用于前端节点为例，该方法可以包括以下步骤：
步骤401，根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；不同的分配组对应不同的子资源池。如根据预设时间窗内接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组。
步骤402，根据分配组中的查询请求的特征信息获得该分配组的资源开销。
步骤403，根据该分配组的资源开销和该分配组对应的子资源池的计算节点资源，动态调节该子资源池中的计算节点。
步骤404，通过该子资源池中的计算节点查询与该分配组中的查询请求对应的数据，也就是说，不同的查询请求可能分配到不同的子资源池中的计算节点。
在一个例子中，上述执行顺序只是为了方便描述给出的一个示例，在实际应用中，还可以改变步骤之间的执行顺序，对此执行顺序不做限制。而且，在其它实施例中，并不一定按照本说明书示出和描述的顺序来执行相应方法的步骤，其方法所包括的步骤可以比本说明书所描述的更多或更少。此外，本说明书中所描述的单个步骤，在其它实施例中可能被分解为多个步骤进行描述；本说明书中所描述的多个步骤，在其它实施例也可能被合并为单个步骤进行描述。
在执行步骤401之前，针对接收到的所有查询请求，还可以先获取每个查询请求的特征信息，该特征信息可以包括但不限于以下之一或者任意组合：并发度、查询复杂度、查询数据量、查询时间、资源占用率。其中，对于特征信息的获取方式，可以参见图2所示的流程，在此不再重复赘述。
在步骤401中，根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组，可以包括但不限于：针对接收到的每个查询请求，可以根据该查询请求的特征信息获得该查询请求的预测资源量，并确定该预测资源量所属的资源区间，并将该查询请求划分到该资源区间对应的分配组；其中，不同的分配组可以对应不同的资源区间。
其中，获得查询请求的预测资源量的过程可以参见步骤201，在此不再赘述。
其中，确定该预测资源量所属的资源区间，并将该查询请求划分到该资源区间对应的分配组，可以包括但不限于：预先为每个子资源池配置资源区间，对此配置方式不做限制，例如，当子资源池的计算节点资源越大时，该子资源池的资源区间可以越大，当子资源池的计算节点资源越小时，该子资源池的资源区间可以越小。例如，子资源池1的计算节点资源为4个CPU核，子资源池2的计算节点资源为8个CPU核，子资源池3的计算节点资源为16个CPU核，则子资源池1的资源区间为[0-1)个CPU核，子资源池2的资源区间为[1-2)个CPU核，子资源池3的资源区间为[2-无穷大)个CPU核。此外，还可以为每个资源区间配置一个分配组，如为子资源池1的资源区间配置分配组1，为子资源池2的资源区间配置分配组2，为子资源池3的资源区间配置分配组3。显然，分配组1对应子资源池1，分配组2对应子资源池2，分配组3对应子资源池3。
进一步的，假设查询请求的预测资源量为1个CPU核，则可以确定该预测资源量所属的资源区间为子资源池2的资源区间，并可以将该查询请求划分到分配组2。显然， 在对预设时间窗内接收到的所有查询请求进行上述处理后，这些查询请求就可以被划分到各个分配组，如查询请求1-10被划分到分配组1，查询请求11-50被划分到分配组2，查询请求51-100被划分到分配组3。
在步骤402中，根据分配组中的查询请求的特征信息获得该分配组的资源开销，可以包括：针对分配组中的每个查询请求，根据该查询请求的特征信息获得该查询请求的预测资源量，并根据该预测资源量获得分配组的资源开销。
其中，步骤402的实现过程可以参见步骤201，不同之处在于：在步骤201中，是针对接收到的所有查询请求进行处理，而步骤402中，是针对分配组中的所有查询请求进行处理，而其它过程类似，在此不再重复赘述。
在步骤403中，根据分配组的资源开销和该分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点，可以包括：根据分配组的资源开销和该分配组对应的子资源池的计算节点资源，获得该子资源池中的计算节点数量；在该子资源池中分配与该计算节点数量匹配的计算节点。
进一步的，在该子资源池中分配与该计算节点数量匹配的计算节点，可以包括：若该子资源池中已经存在的计算节点的数量小于该计算节点数量，则在该子资源池中扩容计算节点，扩容后的计算节点的数量大于等于计算节点数量；若该子资源池中已经存在的计算节点的数量大于计算节点数量，则在该子资源池中缩容计算节点，缩容后的计算节点的数量大于等于该计算节点数量。
其中，步骤403的实现过程可以参见步骤202，不同之处在于：在步骤202中，是根据接收到的所有查询请求的资源开销和计算节点资源，动态调节资源池中的计算节点，而步骤403中，是根据分配组的资源开销和该分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点。
例如，针对步骤403，可以根据分配组1的资源开销和子资源池1的计算节点资源，获得子资源池1中的计算节点数量10，并在子资源池1中分配10个计算节点。此外，可以根据分配组2的资源开销和子资源池2的计算节点资源，获得子资源池2中的计算节点数量8，并在子资源池1中分配8个计算节点。此外，可以根据分配组3的资源开销和子资源池3的计算节点资源，获得子资源池3中的计算节点数量13，并在子资源池3中分配13个计算节点。
其中，步骤404的实现过程可以参见步骤203，不同之处在于：在步骤203中，是将查询请求对应的查询请求发送给资源池的计算节点，步骤404中，是将分配组1的查 询请求对应的查询请求发送给子资源池1的计算节点，将分配组2的查询请求对应的查询请求发送给子资源池2的计算节点，将分配组3的查询请求对应的查询请求发送给子资源池3的计算节点，在此不再重复赘述。
基于与上述方法同样的申请构思，本申请实施例还提供一种数据查询装置，如图5所示，为该装置的结构图，该装置包括：
获得模块501，用于根据接收到的查询请求的特征信息获得资源开销；处理模块502，根据资源开销和计算节点资源动态调节资源池中的计算节点；查询模块503，用于通过所述计算节点查询与所述查询请求对应的数据。
在一个例子中，所述获得模块501还用于：当特征信息包括查询复杂度时，从查询请求中获取查询关键字；通过所述查询关键字查询第一映射表，得到与所述查询关键字对应的复杂度值，将所述复杂度值确定为所述查询请求对应的查询复杂度；或者，从查询请求的多个子查询中获取查询关键字；通过获取的查询关键字查询第一映射表，得到与查询关键字对应的复杂度值；将得到的复杂度值之和确定为所述查询请求对应的查询复杂度；其中，所述第一映射表用于记录查询关键字与复杂度值的对应关系。
基于与上述方法同样的构思，本申请实施例提供一种数据查询设备，包括处理器，用于根据接收到的查询请求的特征信息获得资源开销；根据所述资源开销和计算节点资源动态调节资源池中的计算节点；通过所述计算节点查询与所述查询请求对应的数据。
基于与上述方法同样的申请构思，本申请实施例还提供一种机器可读存储介质，可以应用于数据查询设备，机器可读存储介质上存储有若干计算机指令；其中，所述计算机指令被执行时进行如下处理：根据接收到的查询请求的特征信息获得资源开销；根据所述资源开销和计算节点资源动态调节资源池中的计算节点；通过所述计算节点查询与所述查询请求对应的数据。
基于与上述方法同样的申请构思，本申请实施例还提供一种数据查询装置，如图6所示，为该装置的结构图，该装置包括：
划分模块601，用于根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；不同分配组对应不同子资源池；获得模块602，用于根据分配组中的查询请求的特征信息获得所述分配组的资源开销；处理模块603，用于根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点；查询模块604，用于通过所述子资源池中的计算节点查询与所述分配组中的查询请求对应的数据。
在一个例子中，所述划分模块603具体用于：针对接收到的查询请求，根据该查询请求的特征信息获得该查询请求的预测资源量，并确定该预测资源量所属的资源区间；将该查询请求划分到所述资源区间对应的分配组；其中，不同的分配组对应不同的资源区间。
基于与上述方法同样的构思，本申请实施例提供一种数据查询设备，包括处理器，用于根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；其中，不同的分配组对应不同的子资源池；根据分配组中的查询请求的特征信息获得所述分配组的资源开销；根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点；通过子资源池中的计算节点查询与所述分配组中的查询请求对应的数据。
基于与上述方法同样的申请构思，本申请实施例还提供一种机器可读存储介质，可以应用于数据查询设备，机器可读存储介质上存储有若干计算机指令；其中，所述计算机指令被执行时进行如下处理：根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；其中，不同的分配组对应不同的子资源池；根据分配组中的查询请求的特征信息获得所述分配组的资源开销；根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点；通过子资源池中的计算节点查询与所述分配组中的查询请求对应的数据。
上述实施例阐明的系统、装置、模块或单元，具体可以由计算机芯片或实体实现，或者由具有某种功能的产品来实现。一种典型的实现设备为计算机，计算机的具体形式可以是个人计算机、膝上型计算机、蜂窝电话、相机电话、智能电话、个人数字助理、媒体播放器、导航设备、电子邮件收发设备、游戏控制台、平板计算机、可穿戴设备或者这些设备中的任意几种设备的组合。
为了描述的方便，描述以上装置时以功能分为各种单元分别描述。当然，在实施本申请时可以把各单元的功能在同一个或多个软件和/或硬件中实现。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程 图和/或方框图来描述的。应理解可以由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其它可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其它可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
而且，这些计算机程序指令也可以存储在能引导计算机或其它可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或者多个流程和/或方框图一个方框或者多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其它可编程数据处理设备上，使得在计算机或者其它可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其它可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
以上所述仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。

Claims (29)

  1. 一种数据查询方法，其特征在于，所述方法包括：
    根据接收到的查询请求的特征信息获得资源开销；
    根据所述资源开销和计算节点资源动态调节资源池中的计算节点；
    通过所述计算节点查询与所述查询请求对应的数据。
  2. 根据权利要求1所述的方法，其特征在于，所述特征信息包括以下之一或者任意组合：并发度、查询复杂度、查询数据量、查询时间、资源占用率。
  3. 根据权利要求1所述的方法，其特征在于，所述根据接收到的查询请求的特征信息获得资源开销之前，所述方法还包括：
    若所述特征信息包括查询复杂度，从查询请求中获取查询关键字；
    通过所述查询关键字查询第一映射表，得到与所述查询关键字对应的复杂度值，并将所述复杂度值确定为所述查询请求对应的查询复杂度；
    其中，所述第一映射表用于记录查询关键字与复杂度值的对应关系。
  4. 根据权利要求1所述的方法，其特征在于，所述根据接收到的查询请求的特征信息获得资源开销之前，所述方法还包括：
    若所述特征信息包括查询复杂度，则从查询请求的多个子查询中获取查询关键字；通过获取的查询关键字查询第一映射表，得到与查询关键字对应的复杂度值；将得到的复杂度值之和确定为所述查询请求对应的查询复杂度；
    其中，所述第一映射表用于记录查询关键字与复杂度值的对应关系。
  5. 根据权利要求3或4所述的方法，其特征在于，
    所述查询请求包括：结构化查询语言SQL查询请求；所述查询关键字包括以下之一或者任意组合：加入关键字、对结果集进行分组的关键字、对结果集进行排序的关键字、列出不同关键字、行数计算关键字、窗口函数关键字。
  6. 根据权利要求1所述的方法，其特征在于，所述根据接收到的查询请求的特征信息获得资源开销之前，所述方法还包括：
    通过查询请求的数据标识查询第二映射表，得到与所述数据标识对应的特征信息；其中，所述第二映射表用于记录数据标识与特征信息的对应关系；所述特征信息包括查询数据量、查询时间、资源占用率中的一个或者多个。
  7. 根据权利要求6所述的方法，其特征在于，所述通过查询请求的数据标识查询第二映射表，得到与所述数据标识对应的特征信息之前，还包括：
    收集历史数据，根据所述历史数据获取数据标识与特征信息的对应关系；
    在所述第二映射表中记录数据标识与特征信息的对应关系。
  8. 根据权利要求1所述的方法，其特征在于，
    所述根据接收到的查询请求的特征信息获得资源开销，包括：
    针对接收到的查询请求，根据该查询请求的特征信息获得该查询请求的预测资源量，并根据查询请求的预测资源量确定资源开销。
  9. 根据权利要求8所述的方法，其特征在于，
    所述根据该查询请求的特征信息获得该查询请求的预测资源量，包括：
    通过预测模型对该查询请求的特征信息进行分析，得到该查询请求的预测资源量；其中，所述预测模型包括：三次指数平滑法Holt-Winter季节模型、自回归与滑动平均ARMA模型、线性回归模型、神经网络模型。
  10. 根据权利要求1所述的方法，其特征在于，所述根据所述资源开销和计算节点资源动态调节资源池中的计算节点，包括：
    根据所述资源开销和计算节点资源获得计算节点数量；
    在资源池中分配与所述计算节点数量匹配的计算节点。
  11. 根据权利要求10所述的方法，其特征在于，
    所述在资源池中分配与所述计算节点数量匹配的计算节点，包括：
    若资源池中已经存在的计算节点的数量小于所述计算节点数量，则在资源池中扩容计算节点，扩容后的计算节点的数量大于等于所述计算节点数量；
    若资源池中已经存在的计算节点的数量大于所述计算节点数量，则在资源池中缩容计算节点，缩容后的计算节点的数量大于等于所述计算节点数量。
  12. 一种数据查询方法，其特征在于，所述方法包括：
    根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；其中，不同的分配组对应不同的子资源池；
    根据分配组中的查询请求的特征信息获得所述分配组的资源开销；
    根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点；
    通过子资源池中的计算节点查询与所述分配组中的查询请求对应的数据。
  13. 根据权利要求12所述的方法，其特征在于，所述特征信息包括以下之一或者任意组合：并发度、查询复杂度、查询数据量、查询时间、资源占用率。
  14. 根据权利要求12所述的方法，其特征在于，
    所述将接收到的查询请求划分到至少一个分配组之前，所述方法还包括：
    若所述特征信息包括查询复杂度，从查询请求中获取查询关键字；
    通过所述查询关键字查询第一映射表，得到与所述查询关键字对应的复杂度值，并将所述复杂度值确定为所述查询请求对应的查询复杂度；
    其中，所述第一映射表用于记录查询关键字与复杂度值的对应关系。
  15. 根据权利要求12所述的方法，其特征在于，
    所述将接收到的查询请求划分到至少一个分配组之前，所述方法还包括：
    若所述特征信息包括查询复杂度，则从查询请求的多个子查询中获取查询关键字；通过获取的查询关键字查询第一映射表，得到与查询关键字对应的复杂度值；将得到的复杂度值之和确定为所述查询请求对应的查询复杂度；
    其中，所述第一映射表用于记录查询关键字与复杂度值的对应关系。
  16. 根据权利要求14或15所述的方法，其特征在于，
    所述查询请求包括：结构化查询语言SQL查询请求；所述查询关键字包括以下之一或者任意组合：加入关键字、对结果集进行分组的关键字、对结果集进行排序的关键字、列出不同关键字、行数计算关键字、窗口函数关键字。
  17. 根据权利要求12所述的方法，其特征在于，
    所述将接收到的查询请求划分到至少一个分配组之前，所述方法还包括：
    通过查询请求的数据标识查询第二映射表，得到与所述数据标识对应的特征信息；其中，所述第二映射表用于记录数据标识与特征信息的对应关系；所述特征信息包括查询数据量、查询时间、资源占用率中的一个或者多个。
  18. 根据权利要求17所述的方法，其特征在于，所述通过查询请求的数据标识查询第二映射表，得到与所述数据标识对应的特征信息之前，还包括：
    收集历史数据，根据所述历史数据获取数据标识与特征信息的对应关系；
    在所述第二映射表中记录数据标识与特征信息的对应关系。
  19. 根据权利要求12所述的方法，其特征在于，根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组，包括：
    针对接收到的查询请求，根据该查询请求的特征信息获得该查询请求的预测资源量，并确定该预测资源量所属的资源区间；将该查询请求划分到所述资源区间对应的分配组；其中，不同的分配组对应不同的资源区间。
  20. 根据权利要求12所述的方法，其特征在于，
    根据分配组中的查询请求的特征信息获得所述分配组的资源开销，包括：
    针对分配组中的查询请求，根据该查询请求的特征信息获得该查询请求的预测资源量，并根据该预测资源量获得所述分配组的资源开销。
  21. 根据权利要求19或20所述的方法，其特征在于，
    所述根据该查询请求的特征信息获得该查询请求的预测资源量，包括：
    通过预测模型对该查询请求的特征信息进行分析，得到该查询请求的预测资源量；其中，所述预测模型包括：三次指数平滑法Holt-Winter季节模型、自回归与滑动平均ARMA模型、线性回归模型、神经网络模型。
  22. 根据权利要求12所述的方法，其特征在于，
    所述根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点，包括：
    根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，获得所述子资源池中的计算节点数量；
    在所述子资源池中分配与所述计算节点数量匹配的计算节点。
  23. 根据权利要求22所述的方法，其特征在于，
    在所述子资源池中分配与所述计算节点数量匹配的计算节点，包括：
    若子资源池中已经存在的计算节点的数量小于所述计算节点数量，则在子资源池中扩容计算节点，扩容后的计算节点的数量大于等于所述计算节点数量；
    若子资源池中已经存在的计算节点的数量大于所述计算节点数量，则在子资源池中缩容计算节点，缩容后的计算节点的数量大于等于所述计算节点数量。
  24. 一种数据查询装置，其特征在于，所述装置包括：
    获得模块，用于根据接收到的查询请求的特征信息获得资源开销；
    处理模块，根据资源开销和计算节点资源动态调节资源池中的计算节点；
    查询模块，用于通过所述计算节点查询与所述查询请求对应的数据。
  25. 根据权利要求24所述的装置，其特征在于，所述获得模块还用于：当特征信息包括查询复杂度时，从查询请求中获取查询关键字；通过所述查询关键字查询第一映射表，得到与所述查询关键字对应的复杂度值，将所述复杂度值确定为所述查询请求对应的查询复杂度；或者，从查询请求的多个子查询中获取查询关键字；通过获取的查询关键字查询第一映射表，得到与查询关键字对应的复杂度值；将得到的复杂度值之和确 定为所述查询请求对应的查询复杂度；其中，所述第一映射表用于记录查询关键字与复杂度值的对应关系。
  26. 一种数据查询装置，其特征在于，应用于前端节点，所述装置包括：
    划分模块，用于根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；其中，不同分配组对应不同的子资源池；
    获得模块，用于根据分配组中的查询请求的特征信息获得所述分配组的资源开销；
    处理模块，用于根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点；
    查询模块，用于通过所述子资源池中的计算节点查询与所述分配组中的查询请求对应的数据。
  27. 根据权利要求26所述的装置，其特征在于，所述划分模块具体用于：针对接收到的查询请求，根据该查询请求的特征信息获得该查询请求的预测资源量，并确定该预测资源量所属的资源区间；将该查询请求划分到所述资源区间对应的分配组；其中，不同的分配组对应不同的资源区间。
  28. 一种数据查询设备，其特征在于，包括：
    处理器，用于根据接收到的查询请求的特征信息获得资源开销；根据所述资源开销和计算节点资源动态调节资源池中的计算节点；通过所述计算节点查询与所述查询请求对应的数据。
  29. 一种数据查询设备，其特征在于，包括：处理器，用于根据接收到的查询请求的特征信息，将接收到的查询请求划分到至少一个分配组；其中，不同的分配组对应不同的子资源池；根据分配组中的查询请求的特征信息获得所述分配组的资源开销；根据所述分配组的资源开销和所述分配组对应的子资源池的计算节点资源，动态调节所述子资源池中的计算节点；通过子资源池中的计算节点查询与所述分配组中的查询请求对应的数据。
PCT/CN2019/078418 2018-03-29 2019-03-18 一种数据查询方法、装置及设备 WO2019184739A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP19775405.4A EP3779688A4 (en) 2018-03-29 2019-03-18 DATA INQUIRY PROCEDURE, SETUP, AND DEVICE
JP2020552239A JP7511477B2 (ja) 2018-03-29 2019-03-18 データクエリ方法、装置、およびデバイス
US17/035,276 US11556541B2 (en) 2018-03-29 2020-09-28 Data query method, apparatus and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810268968.0A CN110321214A (zh) 2018-03-29 2018-03-29 一种数据查询方法、装置及设备
CN201810268968.0 2018-03-29

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/035,276 Continuation US11556541B2 (en) 2018-03-29 2020-09-28 Data query method, apparatus and device

Publications (1)

Publication Number Publication Date
WO2019184739A1 true WO2019184739A1 (zh) 2019-10-03

Family

ID=68060965

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/078418 WO2019184739A1 (zh) 2018-03-29 2019-03-18 一种数据查询方法、装置及设备

Country Status (5)

Country Link
US (1) US11556541B2 (zh)
EP (1) EP3779688A4 (zh)
JP (1) JP7511477B2 (zh)
CN (1) CN110321214A (zh)
WO (1) WO2019184739A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110909048A (zh) * 2019-12-03 2020-03-24 北京明略软件系统有限公司 数据查询方法、装置、服务器、客户端及存储介质
CN111930523A (zh) * 2020-09-28 2020-11-13 支付宝(杭州)信息技术有限公司 一种用于服务集群的负载均衡方法和系统
CN112749174A (zh) * 2019-10-30 2021-05-04 中国移动通信集团安徽有限公司 高并发处理方法、装置、处理设备及计算机存储介质
EP4155969A1 (en) * 2021-09-27 2023-03-29 Beijing Baidu Netcom Science And Technology Co. Ltd. Query processing method and apparatus, electronic device and storage medium
US12124454B2 (en) 2020-08-04 2024-10-22 International Business Machines Corporation Shadow experiments for serverless multi-tenant cloud services

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111400555B (zh) * 2020-03-05 2023-09-26 湖南大学 图数据查询任务处理方法、装置、计算机设备和存储介质
CN111767252A (zh) * 2020-06-30 2020-10-13 平安科技（深圳）有限公司 日志查询方法、装置、计算机设备和存储介质
US12063150B1 (en) * 2020-08-11 2024-08-13 Amazon Technologies, Inc. Quality of service management for preventing client drop-offs
WO2022041143A1 (en) * 2020-08-28 2022-03-03 Alibaba Group Holding Limited Smart procedure routing in partitioned database management systems
CN113297027A (zh) * 2020-08-31 2021-08-24 阿里巴巴集团控股有限公司 选择计算节点的方法、装置以及数据库
US11809424B2 (en) * 2020-10-23 2023-11-07 International Business Machines Corporation Auto-scaling a query engine for enterprise-level big data workloads
CN113742378A (zh) * 2021-01-15 2021-12-03 北京沃东天骏信息技术有限公司 数据查询及存储方法、相关设备及存储介质
US12019631B1 (en) * 2022-03-23 2024-06-25 Wells Fargo Bank, N.A. Systems and methods for reducing computational resource usage in a query-based data access system via a repeated query results repository
CN115328972B (zh) * 2022-08-24 2023-05-23 桂林电子科技大学 一种平滑自回归基数估计方法
CN115567537B (zh) * 2022-09-20 2024-06-21 中国联合网络通信集团有限公司 一种资源的调度方法和设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102360314A (zh) * 2011-10-28 2012-02-22 中国科学院计算技术研究所 一种数据中心资源管理系统和方法
US20160188594A1 (en) * 2014-12-31 2016-06-30 Cloudera, Inc. Resource management in a distributed computing environment
CN107580023A (zh) * 2017-08-04 2018-01-12 山东大学 一种动态调整任务分配的流处理作业调度方法及系统

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3716753B2 (ja) 2001-03-21 2005-11-16 日本電気株式会社 マルチプロセッサ構成の計算機間におけるトランザクション負荷分散方法及び方式並びにプログラム
US7594229B2 (en) 2001-10-09 2009-09-22 Nvidia Corp. Predictive resource allocation in computing systems
CA2744925C (en) 2003-04-08 2014-06-03 Grant L. Hutchison Method and system for executing a database query
US20050055694A1 (en) * 2003-09-04 2005-03-10 Hewlett-Packard Development Company, Lp Dynamic load balancing resource allocation
US8555288B2 (en) 2006-05-17 2013-10-08 Teradata Us, Inc. Managing database utilities to improve throughput and concurrency
US8209695B1 (en) 2006-07-28 2012-06-26 Hewlett-Packard Development Company, L.P. Reserving resources in a resource-on-demand system for user desktop utility demand
US7669029B1 (en) 2006-11-15 2010-02-23 Network Appliance, Inc. Load balancing a data storage system
US9164689B2 (en) 2009-03-30 2015-10-20 Oracle America, Inc. Data storage system and method of processing a data access request
US8965860B2 (en) 2010-04-01 2015-02-24 Salesforce.Com, Inc. Methods and systems for bulk uploading of data in an on-demand service environment
US8849749B2 (en) 2010-05-14 2014-09-30 Oracle International Corporation Load balancing in parallel database systems using multi-reordering
US8479211B1 (en) * 2010-06-29 2013-07-02 Amazon Technologies, Inc. Dynamic resource commitment management
US20130205028A1 (en) * 2012-02-07 2013-08-08 Rackspace Us, Inc. Elastic, Massively Parallel Processing Data Warehouse
US9489222B2 (en) 2011-08-24 2016-11-08 Radware, Ltd. Techniques for workload balancing among a plurality of physical machines
WO2016003334A1 (en) 2014-07-01 2016-01-07 Telefonaktiebolaget L M Ericsson (Publ) Predicting resource scheduling
US20160063029A1 (en) * 2014-08-29 2016-03-03 Netapp, Inc. Clustered storage system synchronization
CN105653572A (zh) * 2015-08-20 2016-06-08 乐视网信息技术（北京）股份有限公司 一种资源的处理方法及装置
US10762539B2 (en) * 2016-01-27 2020-09-01 Amobee, Inc. Resource estimation for queries in large-scale distributed database system
CN107168977B (zh) * 2016-03-08 2020-07-28 阿里巴巴集团控股有限公司 一种数据查询的优化方法及装置
US10853367B1 (en) * 2016-06-16 2020-12-01 Intuit Inc. Dynamic prioritization of attributes to determine search space size of each term, then index on those sizes as attributes
US10762086B2 (en) * 2016-09-01 2020-09-01 Amazon Technologies, Inc. Tracking query execution status for selectively routing queries
US10108459B2 (en) * 2016-09-12 2018-10-23 Bmc Software, Inc. System and method to dynamically allocate varying processing capacity entitlements based on workload importance
US11010205B2 (en) * 2017-05-30 2021-05-18 Hewlett Packard Enterprise Development Lp Virtual network function resource allocation

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102360314A (zh) * 2011-10-28 2012-02-22 中国科学院计算技术研究所 一种数据中心资源管理系统和方法
US20160188594A1 (en) * 2014-12-31 2016-06-30 Cloudera, Inc. Resource management in a distributed computing environment
CN107580023A (zh) * 2017-08-04 2018-01-12 山东大学 一种动态调整任务分配的流处理作业调度方法及系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3779688A4

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112749174A (zh) * 2019-10-30 2021-05-04 中国移动通信集团安徽有限公司 高并发处理方法、装置、处理设备及计算机存储介质
CN112749174B (zh) * 2019-10-30 2024-05-10 中国移动通信集团安徽有限公司 高并发处理方法、装置、处理设备及计算机存储介质
CN110909048A (zh) * 2019-12-03 2020-03-24 北京明略软件系统有限公司 数据查询方法、装置、服务器、客户端及存储介质
US12124454B2 (en) 2020-08-04 2024-10-22 International Business Machines Corporation Shadow experiments for serverless multi-tenant cloud services
CN111930523A (zh) * 2020-09-28 2020-11-13 支付宝(杭州)信息技术有限公司 一种用于服务集群的负载均衡方法和系统
EP4155969A1 (en) * 2021-09-27 2023-03-29 Beijing Baidu Netcom Science And Technology Co. Ltd. Query processing method and apparatus, electronic device and storage medium

Also Published As

Publication number Publication date
US11556541B2 (en) 2023-01-17
EP3779688A1 (en) 2021-02-17
JP7511477B2 (ja) 2024-07-05
US20210011916A1 (en) 2021-01-14
JP2021519460A (ja) 2021-08-10
CN110321214A (zh) 2019-10-11
EP3779688A4 (en) 2022-01-05

Similar Documents

Publication Publication Date Title
WO2019184739A1 (zh) 一种数据查询方法、装置及设备
US11888702B2 (en) Intelligent analytic cloud provisioning
US11734271B2 (en) Data query method, apparatus and device
CN111722806B (zh) 云盘分配方法、装置、电子设备及存储介质
RU2675054C2 (ru) Балансировка нагрузки для больших баз данных в оперативной памяти
CN113821311A (zh) 任务执行方法及存储设备
KR20150114965A (ko) 낮은 지연속도 데이터 액세스를 위한 데이터 스트림의 분할
US9256506B1 (en) System and method for performing operations on target servers
WO2013078583A1 (zh) 优化数据访问的方法及装置、优化数据存储的方法及装置
US9817698B2 (en) Scheduling execution requests to allow partial results
Senthilkumar et al. A survey on job scheduling in big data
CN111400301B (zh) 一种数据查询方法、装置及设备
Shabeera et al. Optimising virtual machine allocation in MapReduce cloud for improved data locality
CN109218385A (zh) 处理数据的方法和装置
WO2016092604A1 (ja) データ処理システムおよびデータアクセス方法
CN118227337B (zh) 一种键值对存储系统的处理方法和装置
US20140214826A1 (en) Ranking method and system
WO2020211718A1 (zh) 一种数据处理方法、装置及设备
Li et al. Resource scheduling approach for multimedia cloud content management
CN108932258A (zh) 数据索引处理方法及装置
CN110909072B (zh) 一种数据表建立方法、装置及设备
CN115114012B (zh) 一种任务分配方法、装置、电子设备及存储介质
CN114443686A (zh) 一种基于关系型数据的压缩图构建方法及装置
KR101512647B1 (ko) 질의처리엔진을 선택하는 방법
CN114296965A (zh) 特征检索方法、装置、电子设备及计算机存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19775405

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020552239

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2019775405

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2019775405

Country of ref document: EP

Effective date: 20201029