CN107679093B - Data query method and device - Google Patents
Data query method and device Download PDFInfo
- Publication number
- CN107679093B CN107679093B CN201710801101.2A CN201710801101A CN107679093B CN 107679093 B CN107679093 B CN 107679093B CN 201710801101 A CN201710801101 A CN 201710801101A CN 107679093 B CN107679093 B CN 107679093B
- Authority
- CN
- China
- Prior art keywords
- metadata
- target
- query
- data
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/144—Query formulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention provides a data query method and a data query device, which are used for solving the technical problem of high development cost of a client in the prior art. The method comprises the following steps: the intermediate layer receives a query request sent by a client, wherein the query request comprises a query condition and a storage format of target data; wherein the query condition is used for defining a constraint condition of metadata of the target data; the intermediate layer acquires target metadata meeting the query condition, and determines a target read command for reading the target data according to the mapping between the storage format of the data and the read command; and the middle layer reads data corresponding to the target metadata from a storage layer according to the determined target reading command and sends the data corresponding to the target metadata to the client, wherein the data corresponding to the target metadata comprises the target data.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a data query method and a data query device.
Background
In the existing distributed storage system, the storage format of the data in the storage layer may be of various types, such as a queue format, an orc format, an RCFile format, and the like, and when reading the target data in the storage layer, the client must use a program code adapted to the storage format of the target data to successfully read the target data. In order to realize the access of a client to data in different storage formats of a storage layer, the prior art is to: a set of adaptive program codes is developed for the client aiming at each storage format, and the client switches to use different types of program codes when reading target data in different storage formats, so that the development cost of the client is high.
Disclosure of Invention
The invention provides a data query method and a data query device, which are used for solving the technical problems that in the prior art, a client needs to switch different types of program codes when reading target data in different storage formats, and the development cost of the client is high.
A first aspect of an embodiment of the present invention provides a data query method, including:
the intermediate layer receives a query request sent by a client, wherein the query request comprises a query condition and a storage format of target data; wherein the query condition is used for defining a constraint condition of metadata of the target data;
the intermediate layer obtains target metadata satisfying the query condition, an
Determining a target read command for reading the target data according to the mapping between the storage format of the data and the read command;
and the middle layer reads data corresponding to the target metadata from a storage layer according to the determined target reading command and sends the data corresponding to the target metadata to the client, wherein the data corresponding to the target metadata comprises the target data.
In the scheme, the client side sends a query request to the middle layer according to a preset protocol, the middle layer reads data corresponding to the query request from the storage layer after receiving the query request sent by the client side according to the preset protocol, and then the data corresponding to the query request is returned to the client side, so that the client side can obtain target data from the storage layer. The access of the client to the data in the storage layer is indirectly realized through the middle layer, which is different from the direct interaction mode of the client and the storage layer in the prior art, the scheme can not switch the types of program codes for the client when inquiring the target data in different storage formats in the storage layer, and can not develop a set of adaptive program codes for the client aiming at each storage format in the stage of developing the client, thereby reducing the development cost of the client.
Optionally, the obtaining, by the intermediate layer, target metadata that meets the query condition includes: the middle layer acquires metadata used for indicating a file where the target data is located; or the middle layer acquires metadata used for indicating a row group where the target data is located; or the middle layer acquires metadata used for indicating a page (page) where the target data is located. Through the method, the middle layer can position the position of the target data according to the acquired target metadata, the method is different from the prior art that the position of the target data can only be positioned to a row group level at most, the method can position the target data to a page level accurately, the loading capacity of the data and the invalid calculation capacity of a client can be effectively reduced, and the data query efficiency is improved.
Optionally, the obtaining, by the intermediate layer, target metadata that meets the query condition includes: the middle layer acquires target metadata meeting the query condition from the storage layer; and/or the middle layer acquires target metadata meeting the query condition from the metadata stored in the middle layer. By the method, the intermediate layer can acquire the target metadata meeting the query condition, so that the access of the client to the data in the storage layer can be indirectly realized through the intermediate layer, a set of adaptive program codes does not need to be developed for the client aiming at each storage format in the stage of developing the client, and the development cost of the client is reduced.
Optionally, the method further includes: the middle layer receives metadata updating information sent by the client; the metadata updating information is generated by performing statistical analysis on data corresponding to the target metadata after the client receives the data corresponding to the target metadata from the middle layer; the middle layer updates the metadata stored in the middle layer according to the metadata updating information; and/or the middle layer updates the metadata in the storage layer according to the metadata updating information. By the method, the middle layer can continuously update the original metadata and continuously expand and supplement the metadata in the process of receiving the client query request to perform data query, so that the metadata can describe more detailed data, the middle layer can acquire more accurate data when querying the target data from the storage layer, the data loading amount and the invalid calculation amount of the client can be effectively reduced, and the data query efficiency is improved.
Optionally, the type of the target metadata includes at least one of a numeric type, a time type, a character type, and an enumeration type. By the method, the middle layer can acquire more types of metadata when the target data is queried from the storage layer, the filtering condition is further refined, the data loading capacity and the invalid calculation capacity of the client side are further reduced, and the data query efficiency is improved.
Optionally, the query condition includes a constraint condition of metadata of the target data and a range of the file to be queried; the intermediate layer acquires target metadata meeting the query condition, and the method comprises the following steps: and the middle layer determines a file to be queried according to the range of the file to be queried, determines metadata of the file to be queried, and screens the metadata meeting the constraint condition from the metadata of the file to be queried as the target metadata. By the method, the middle layer can position the position of the target data according to the obtained target metadata, so that the data loading amount and the invalid calculation amount of the client are further reduced, and the data query efficiency is improved.
Optionally, an inquiry cache is stored in the middle layer, where the inquiry cache includes at least one item of historical inquiry request and target metadata corresponding to each item in the at least one item of historical inquiry request; after the middle layer receives the query request sent by the client, the method comprises the following steps: the intermediate layer judges whether the query cache comprises the query request or not; if so, the intermediate layer acquires the target metadata corresponding to the query request in the query cache. By the method, when the query request is one of the historical query requests, the middle layer can directly determine the target metadata according to the query cache stored before, the metadata screening process is not needed, and the data query efficiency is further improved.
Optionally, the method further includes: the intermediate layer adds a query request received in a preset time range and target metadata corresponding to the query request received in the preset time range in the query cache; and/or the middle layer adds the query requests with the request times exceeding the preset times and the target metadata corresponding to the query requests with the request times exceeding the preset times in the query cache. Through the method, the recent historical query request and the target metadata corresponding to the recent historical query request, the high-frequency query request and the target metadata corresponding to the high-frequency query request can be cached, so that the intermediate layer can directly determine the target metadata according to the previously stored query cache when receiving the recent query request or the high-frequency query request, the metadata screening process is not needed, and the data query efficiency is further improved.
Optionally, an inquiry cache is stored in the middle layer, where the inquiry cache includes at least one item of historical inquiry request and target data corresponding to each item in the at least one item of historical inquiry request; the intermediate layer acquires target data meeting the query condition, and the method comprises the following steps: the intermediate layer judges whether the query cache comprises the query request or not; if so, the intermediate layer acquires the target data corresponding to the query request in the query cache. By the method, when the query request is one of the historical query requests, the intermediate layer can directly determine the target data according to the query cache stored before, a metadata screening process and a process of reading data corresponding to the target metadata from the storage layer are not needed, and the data query efficiency is further improved.
Optionally, the receiving, by the intermediate layer, the query request sent by the client includes: and the intermediate layer receives the query request sent by the client according to a preset protocol, wherein the preset protocol is a protocol which is followed by different clients and the intermediate layer when the different clients communicate. By the method, different clients can communicate with the middle layer according to the preset protocol, so that the access of the clients to the data in the storage layer can be indirectly realized through the middle layer, a set of adaptive program codes does not need to be developed for the clients according to each storage format in the client development stage, and the client development cost is reduced.
Optionally, the intermediate layer may include the following three functional modules: the device comprises a metadata storage module, a metadata updating module and a query cache module. The metadata storage module is used for acquiring and storing metadata of files in a storage layer, the metadata updating module is used for updating the metadata in the metadata storage module or updating the metadata in the storage layer according to metadata updating information sent by a client, and the updating mode can specifically comprise two modes of further supplementing original metadata and expanding new metadata; the query cache module is used for caching the historical query requests sent by the client and target metadata or target data corresponding to the historical query requests. Through the intermediate layer provided by the method, the technical effects of saving the development cost of the client, reducing the loading capacity of data query and the invalid calculation capacity of the client and improving the data query efficiency can be achieved.
A second aspect of the embodiments of the present invention further provides a data requesting method, including:
a client sends a query request to an intermediate layer, wherein the query request comprises query conditions and a storage format of target data, the query conditions are used for defining constraint conditions of metadata of the target data so as to enable the intermediate layer to obtain the target metadata meeting the query conditions, a target read command for reading the target data is determined according to the mapping between the storage format of the data and the read command, the data corresponding to the target metadata is read from a storage layer according to the determined target read command, and the data corresponding to the target metadata is sent to the client; wherein the data corresponding to the target metadata comprises the target data; and the client receives the data corresponding to the target metadata sent by the middle layer.
Optionally, the sending, by the client, the query request to the middle layer includes: and the client sends the query request to the client according to a preset protocol, wherein the preset protocol is a protocol which is followed by different clients and the intermediate layer for communication.
Optionally, after the client receives the data corresponding to the target metadata sent by the middle layer, the method further includes: the client carries out statistical analysis on data corresponding to the target metadata to generate metadata updating information; and sending the metadata updating information to the middle layer so that the middle layer updates the metadata stored in the middle layer according to the metadata updating information and/or updates the metadata in the storage layer according to the metadata updating information.
A third aspect of the embodiments of the present invention provides a data query apparatus, including: the system comprises a receiving unit, a sending unit and a processing unit, wherein the receiving unit is used for receiving a query request sent by a client, and the query request comprises a query condition and a storage format of target data; wherein the query condition is used for defining a constraint condition of metadata of the target data; the determining unit is used for acquiring target metadata meeting the query condition and determining a target read command for reading the target data according to the mapping between the storage format of the data and the read command; the reading unit is used for reading data corresponding to the target metadata from the storage layer according to the determined target reading command; and the sending unit is used for sending the data corresponding to the target metadata to the client, and the data corresponding to the target metadata comprises the target data.
Optionally, the determining unit is configured to: acquiring metadata used for indicating a file where the target data is located; or acquiring metadata used for indicating a row group where the target data is located; or acquiring metadata used for indicating the page where the target data is located.
Optionally, the determining unit is configured to: acquiring target metadata meeting the query condition from the storage layer; and/or obtaining target metadata satisfying the query condition from metadata stored by the device.
Optionally, the receiving unit is further configured to: receiving metadata updating information sent by the client; the metadata updating information is generated by performing statistical analysis on data corresponding to the target metadata after the client receives the data corresponding to the target metadata from the device; the apparatus further comprises an update unit configured to: updating the metadata stored by the device according to the metadata updating information, and/or updating the metadata in the storage layer according to the metadata updating information.
Optionally, the type of the target metadata includes at least one of a numeric type, a time type, a character type, and an enumeration type.
Optionally, the apparatus stores a query cache, where the query cache includes at least one item of historical query request and target metadata corresponding to each item in the at least one item of historical query request; the determination unit is configured to: and judging whether the query cache comprises the query request or not, and acquiring target metadata corresponding to the query request in the query cache when the query cache comprises the query request.
Optionally, the apparatus further includes a cache unit, configured to: adding a query request received in a preset time range and target metadata corresponding to the query request received in the preset time range in the query cache; and/or adding query requests with the request times exceeding the preset times and target metadata corresponding to the query requests with the request times exceeding the preset times in the query cache.
Optionally, the receiving unit is configured to: and receiving the query request sent by the client according to a predetermined protocol, wherein the predetermined protocol is a protocol which is followed by different clients and the device to communicate.
The fourth aspect of the embodiments of the present invention further provides a client, including: a sending unit, configured to send a query request to an intermediate layer, where the query request includes a query condition and a storage format of target data, where the query condition is used to define a constraint condition of metadata of the target data, so that the intermediate layer obtains the target metadata that meets the query condition, determines a target read command for reading the target data according to a mapping between the storage format of the data and the read command, reads data corresponding to the target metadata from a storage layer according to the determined target read command, and sends the data corresponding to the target metadata to the client; wherein the data corresponding to the target metadata comprises the target data; and the receiving unit is used for receiving the data corresponding to the target metadata sent by the middle layer.
Optionally, the client further includes: the processing unit is used for performing statistical analysis on the data corresponding to the target metadata after the receiving unit receives the data corresponding to the target metadata sent by the middle layer, and generating metadata updating information; the sending unit is further configured to: and sending the metadata updating information to the middle layer so that the middle layer updates the metadata stored in the middle layer according to the metadata updating information and/or updates the metadata in the storage layer according to the metadata updating information.
Optionally, the sending unit is configured to: and sending the query request to the middle layer according to a preset protocol, wherein the preset protocol is a protocol which is followed by different clients and the middle layer to communicate.
The fifth aspect of the embodiments of the present invention further provides a data query device, including: the system comprises at least one processor, a memory and a communication interface, wherein the memory and the communication interface are in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor executes the data query method provided by the first aspect of the embodiments of the present invention by executing the instructions stored in the memory using the communication interface.
The sixth aspect of the present invention further provides a client, including: the system comprises at least one processor, a memory and a communication interface, wherein the memory and the communication interface are in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor performs the method for requesting data provided by the second side of the embodiment of the present invention using the communication interface by executing the instructions stored in the memory.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a diagram of a distributed query system architecture in an embodiment of the invention;
FIG. 2 is a flowchart illustrating a data query method according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating an intermediate layer updating metadata according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a data query device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a client according to an embodiment of the present invention.
Detailed Description
The technical solutions of the present invention are described in detail below with reference to the drawings and the specific embodiments, and it should be understood that the specific features in the embodiments and the embodiments of the present invention are not intended to limit the technical solutions of the present invention, but may be combined with each other without conflict.
The data query method provided by the embodiment of the invention can be applied to a distributed storage system, and the distributed storage system can be a column type storage system and can also be a line type storage system. Fig. 1 is a schematic diagram of a distributed query system architecture provided in an embodiment of the present invention, and a technical solution in the embodiment of the present invention improves an original distributed query system architecture, specifically, adds an intermediate layer between a client and a storage layer, so that the client accesses data in the storage layer through the intermediate layer.
The client in the embodiment of the present invention may be a distributed computing engine installed in a server, a personal computer, a tablet computer, or a network device such as a smart phone, for example, MapReduce, spark, or the like; the specific implementation of the storage layer in the embodiment of the present invention may be a storage device in a server on each network node in the distributed storage system, or an internal memory or an external memory in a personal computer, a tablet computer, a smart phone, or the like, such as a floppy disk, a hard disk, an optical disk, a usb disk, a mobile hard disk, a read-only memory, a random access memory, or various types of intelligent memories existing as entities individually, which is not limited in the embodiment of the present invention; the data in the storage layer in the embodiment of the present invention may have multiple storage formats, such as a queue format, an orc format, an RCFile format, and the like shown in fig. 1, and the intermediate layer may be implemented as one or more application programs or application software, and the intermediate layer and the storage layer may be in the same entity device or in different entity devices.
In the following description of the embodiment of the present invention, a data query method according to the embodiment of the present invention is described in detail mainly by taking a columnar storage system as an example.
Example one
An embodiment of the present invention provides a data query method, referring to fig. 2, where the method includes:
step 101: the intermediate layer receives a query request sent by a client, wherein the query request comprises a query condition and a storage format of target data.
Specifically, the intermediate layer receives the query request sent by the client according to a predetermined protocol, where the predetermined protocol is a protocol that different clients follow when communicating with the intermediate layer.
Wherein, the query request may specifically include three types of information: reading indication, query condition and storage format of target data. The reading instruction is used for instructing the middle layer to execute the operation of reading the target data from the storage layer; the query condition may include a constraint condition of the metadata of the target data and a range of the file to be queried, and the query condition is used to determine the range of the file to be queried and define the constraint condition of the metadata of the target data; the storage format of the target data is used for determining a target read command for reading the target data. The client, the intermediate layer, and the intermediate layer may be on the same network node or different network nodes, and embodiments of the present invention are not limited specifically.
Step 102: and acquiring target metadata meeting the query condition, and determining a target read command for reading the target data according to the mapping between the storage format of the data and the read command.
After receiving the query request, the intermediate layer can determine the file to be queried according to the range of the file to be queried, then obtain metadata representing the characteristics of the data in the file to be queried, namely the metadata of the file to be queried, and screen the metadata meeting the constraint condition from the metadata of the file to be queried to serve as the target metadata. The middle layer may obtain the target metadata meeting the query condition from the storage layer, or may obtain the target metadata meeting the query condition from the metadata stored in the middle layer, which is not limited in the embodiments of the present invention.
For example: assuming that the storage layer stores 100 files, including file a, file b, file c, file d, etc., wherein the metadata of file a includes a1 and a2, a1 indicates that the size range of the first row group data of file a is [400,730], and a2 indicates that the size range of the second row group data of file a is [750,800 ]; the metadata of the file b comprises b1, b2 and b3, wherein b1 represents that the size range of the first row group data of the file b is [200,400], b2 represents that the size range of the second row group data of the file b is [400,550], and b3 represents that the size range of the third row group data of the file b is [600,750 ]; the files (such as file c, file d, etc.) in the 100 files except for file a and file b do not contain metadata representing data ranges within [100,500 ]. The query conditions received by the middle layer are assumed as follows: querying data with a value larger than 100 and smaller than 500 in the 100 files, the middle layer may first obtain metadata of the 100 files, find that the numerical ranges of the file c, the file d, and the like do not satisfy the query condition, and only the file a and the file b satisfy the query condition, and then locate the query range to the file a and the file c, then further screen the metadata of the file a and the file b, and finally determine that the metadata a1, the metadata b1, and the metadata b2 are target metadata corresponding to the query request.
After the intermediate layer receives the query request, the intermediate layer can determine what type of target read command should be used when the intermediate layer reads the target data from the storage layer according to the storage format of the target data, the storage format of the data stored in the intermediate layer and the mapping of the type of the read command, so that the intermediate layer can read the target data from the storage layer by adopting the read command matched with the storage format of the target data.
Step 103: and the middle layer reads data corresponding to the target metadata from a storage layer according to the determined target reading command and sends the data corresponding to the target metadata to the client, wherein the data corresponding to the target metadata comprises the target data.
Specifically, the intermediate layer generates a target read command according to the determined type of the target read command and the target metadata, then sends the target read command to the storage layer, the storage layer responds to the target read command after receiving the target read command, sends data corresponding to the target metadata to the intermediate layer, and the intermediate layer sends the data corresponding to the target metadata to the client.
For example, following the example in step 102, the intermediate layer determines that the target metadata is b1 and b2, assuming that the storage format of the data in the file b is a parquet format, the intermediate layer generates a target command according to the target metadata and the storage format of the target data, that is, the parquet format, and after the intermediate layer sends the target command to the storage layer, the storage layer sends the data corresponding to the metadata b2 and b3 in the file b, that is, the data in the first row group and the second row group of the file b, to the intermediate layer, and the intermediate layer sends the data in the first row group and the second row group of the file b to the client.
In the scheme, the client side sends a query request to the middle layer according to a preset protocol, the middle layer reads data corresponding to the query request from the storage layer after receiving the query request sent by the client side according to the preset protocol, and then the data corresponding to the query request is returned to the client side, so that the client side can obtain target data from the storage layer. The access of the client to the data in the storage layer is indirectly realized through the middle layer, which is different from the direct interaction mode of the client and the storage layer in the prior art, the scheme can not switch the types of program codes for the client when inquiring the target data in different storage formats in the storage layer, and can not develop a set of adaptive program codes for the client aiming at each storage format in the stage of developing the client, thereby reducing the development cost of the client.
In the scheme of the prior art that a client directly interacts with a storage layer to realize data query, metadata of each file is stored in the file, so that the client must traverse all files to be queried in the storage layer to obtain the metadata of the files to be queried when querying the data, and then determines target metadata meeting query conditions from the obtained metadata of the files to be queried, that is, the client needs to read all the files to be queried in the storage layer before obtaining the target data regardless of the size and range of the queried data, so that the data loading capacity is large, and the data query efficiency is low.
In order to solve the problems of large data loading capacity and low data query efficiency in the prior art, the embodiment of the invention provides a technical scheme for storing metadata in an intermediate layer, which comprises the following steps:
optionally, before the obtaining of the target metadata meeting the query condition in step 102 is performed, the method further includes: the middle layer obtains the metadata from the storage layer and stores the metadata in a unified form, so that after the middle layer receives a query request sent by a client, target metadata meeting the query condition can be searched from the metadata stored in the middle layer according to the query condition in the query request. The intermediate layer may obtain only a part of metadata in the storage layer, or may obtain all metadata in the storage layer, which is not specifically limited in the embodiment of the present invention. By the method, the middle layer can directly acquire the metadata of the file to be inquired from the stored metadata after receiving the inquiry request sent by the client, and can acquire the target metadata without reading the file in the storage layer, so that the data loading amount and the invalid calculation amount of the client can be effectively reduced, and the data inquiry efficiency is improved.
Optionally, the method may further include: and the middle layer receives metadata updating information sent by the client, updates the metadata stored in the middle layer according to the metadata updating information, and/or updates the metadata in the storage layer. The metadata updating information is generated by performing statistical analysis on data corresponding to the target metadata after the client receives the data corresponding to the target metadata from the middle layer.
Specifically, referring to fig. 3, after receiving data corresponding to the target metadata returned by the intermediate layer, the client may perform statistical analysis on characteristics of the data corresponding to the target metadata, for example, analyze a correspondence between a specific distribution range of the data in the storage layer and a numerical value of the data, a correspondence between a specific distribution range of the data in the storage layer and a timestamp of the data, a correspondence between a specific distribution range of the data in the storage layer and a character type of the data, and the like, generate a statistical analysis result, and then determine whether the metadata in the intermediate layer or the storage layer can be processed according to the generated statistical analysis result, if so, the client may generate metadata update information according to the statistical analysis result and send the metadata update information to the intermediate layer, the middle layer can update the original metadata after receiving the metadata updating information. After receiving the metadata update information, the middle layer may update the metadata stored in the middle layer according to the metadata update information, or update the metadata in the storage layer, which is not limited in the embodiments of the present invention.
Wherein the metadata update information may include the following two types:
the 1 st: the original type of metadata is supplemented.
For example, if the query condition is to query data in the file c with a timestamp range of 2016080813: 00-2016080818: 00, but metadata of the timestamp type of the file c, which is acquired by the middle layer after receiving the query request, can only represent the timestamp range of the whole data in the file c, which is 2016080811: 00-2016080819: 00, the middle layer reads the whole data of the file c from the storage layer and returns the whole data to the client, and the client acquires target data meeting the query condition from the whole data of the file c after receiving the data sent by the middle layer, that is, screens out the data with the timestamp range of 2016080813: 00-2016080818: 00. In the process that a client acquires target data from data returned by an intermediate layer, the data in a file c is found to specifically comprise three row groups, the timestamp range of the data in the first row group of the file c is 2016080811: 00-2016080812: 00, the timestamp range of the data in the second row group of the file c is 2016080813: 00-2016080815: 00, and the timestamp range of the data in the third row group of the file c is 2016080816: 00-2016080818: 00, so that the client screens out the data in the second row group and the data in the third row group from the data returned by the intermediate layer as the target data, generates metadata updating information for the file c according to the timestamp characteristics of the row groups of the file c and sends the generated metadata updating information to the intermediate layer, and the intermediate layer updates the original metadata after receiving the metadata updating information to generate the metadata for representing the timestamp range of the first row data, the metadata for representing the timestamp range of the data in the second row group and the metadata for representing the timestamp range of the data in the third row And (4) surrounding metadata, and further refining the metadata of the timestamp type of the file c to the row group level of the file c.
Metadata in the prior art indicates that the existence of data in a file can only range up to the level of a row group, but in the implementation of the present invention, the metadata can be refined to pages in the file. For example, assuming that the query condition is to query data in a file e with a timestamp range of 2016080813: 00-2016080818: 00, after receiving the query request, the middle layer determines that the second row group data (with the timestamp range of 2016080813: 00-2016080815: 00) and the third row group data (with the timestamp range of 2016080816: 00-2016080818: 00) of the file e meet the query condition according to metadata stored in the storage layer or the middle layer, and reads the second row group data and the third row group data of the file e and returns the read data to the client. After the client receives the data returned by the middle layer, in the process of acquiring the target data from the data returned by the middle layer, the timestamp of the data of the second row group of the file e is found to be fault, the timestamp range of the data of the first page in the second row group is 2016080813: 00-2016080814: 00, and the timestamp range of the data of the second page in the second row group is 2016080814: 30-2016080815: 00, then the client can generate metadata update information for the file e according to the timestamp characteristics of the data of the first page and the second page in the second row group of the file e, and sending the generated metadata updating information to the middle layer, generating metadata for representing the timestamp range of the first page data of the second row group and metadata for representing the timestamp range of the second page data of the second row group for the file e by the middle layer after receiving the metadata updating information, and further refines the timestamp type metadata of file e to the page level of file c. For another example, assume that the metadata of the first row group of file f characterizes a size range of data of the first row group of [10,1000 ]. After the client obtains the first row data, the client finds that the data of the first row group of the file f has a fault, specifically [10, 15], [240, 290] and [900,1000], and then the client generates metadata updating information according to the fault characteristics of the row group data, so that the middle layer updates the metadata of the first row group of the file f, and further refines the range of the metadata of the file f.
In the 2 nd, a new type of metadata is extended. Extensible types of metadata include, but are not limited to, the following: time type, numeric type, character type, enumeration type, and the like.
For example, if the query condition is to query data in a file d with a timestamp range of 2016080813: 00-2016080818: 00, the middle layer determines that the second row group data (with the timestamp range of 2016080813: 00-2016080815: 00) and the third row group data (with the timestamp range of 2016080816: 00-2016080818: 00) of the file d meet the query condition according to metadata stored in the storage layer or the middle layer after receiving the query request, and the second row group data and the third row group data of the file d are read out and returned to the client. After the client receives the data returned by the middle layer, in the process of acquiring the target data from the data returned by the middle layer, the client finds that the first letters of the second row group data of the file C are all B, and the first letters of the third row group data of the file C are all C, then the client can generate metadata update information for the file d according to the characteristics of the first letters of the second row group and the third row group of the file C, and sends the generated metadata update information to the intermediate layer, the intermediate layer generates metadata for the character type of the first letter of the second row group data of the file d and metadata for the character type of the third row group data of the file d for the file d after receiving the metadata update information, and further, the metadata type of the file d is expanded to the character type of the data, so that the data in the file d can be inquired according to the initial character of the data in the next data inquiry.
The middle layer receives the metadata updating information sent by the client, updates the metadata stored in the middle layer according to the metadata updating information, and/or updates the metadata in the storage layer, and the middle layer can continuously update the original metadata and continuously expand and supplement the metadata in the process of receiving a client query request to perform data query, so that the metadata can describe more detailed data, the middle layer can acquire more accurate data when querying target data from the storage layer, the data loading amount and the invalid calculation amount of the client are further reduced, and the data query efficiency is improved.
Optionally, with reference to the technical solution that the intermediate layer may update the original metadata, the obtaining of the target metadata that meets the query condition in step 102 in the embodiment of the present invention may specifically include: the middle layer acquires metadata used for indicating a file where the target data is located; or the middle layer acquires metadata used for indicating a row group where the target data is located; or the middle layer acquires metadata used for indicating the page where the target data is located.
Specifically, in the process of acquiring target metadata, the intermediate layer firstly queries metadata used for indicating a file where the target data is located, determines the file where the target data is located, and then further queries whether metadata used for indicating a row group of the target data in the file exists; if not, determining that the target metadata is metadata for indicating the file where the target data is located, and if so, further querying whether metadata for indicating the page of the row group of the target data in the file exists; and if not, determining that the target metadata is metadata for indicating a row group where the target data is located, and if so, determining that the target metadata is metadata for indicating a page where the target data is located. The accuracy of the target metadata indicating the location of the target data depends on the detail of the data features that can be represented by the metadata, and embodiments of the present invention are not limited specifically.
Through the method, the middle layer can position the position of the target data according to the acquired target metadata, the method is different from the prior art that the position of the target data can only be positioned to a page level at most, the method can position the target data to the page level accurately, the loading capacity of the data and the invalid calculation capacity of a client are further reduced, and the data query efficiency is improved.
Optionally, in the embodiment of the present invention, the intermediate layer may record query requests of the client, cache the query records, and generate a query cache in the intermediate layer, where the query cache includes at least one item of historical query requests and target metadata corresponding to each item of the at least one item of historical query requests. The intermediate layer may add, in the query cache, the query request received within the predetermined time range and the target metadata corresponding to the query request received within the predetermined time range, or add, in the query cache, the query request whose request frequency exceeds the predetermined frequency and the target metadata corresponding to the query request whose request frequency exceeds the predetermined frequency.
When the intermediate layer stores the query cache, the implementation manner of the intermediate layer acquiring the target metadata meeting the query condition in step 102 may further include: the intermediate layer judges whether the query cache comprises the query request or not; if so, the intermediate layer acquires the target metadata corresponding to the query request in the query cache, and takes the target metadata acquired from the query cache as the target metadata meeting the query condition.
By the method, when the query request is one of the historical query requests, the intermediate layer can directly determine the target metadata according to the query cache stored before, the metadata screening process is not needed, and the data query efficiency is further improved.
Optionally, in the embodiment of the present invention, the intermediate layer may record query requests of the client, cache the query records, and generate a query cache in the intermediate layer, where the query cache includes at least one item of historical query requests and target data corresponding to each item of the at least one item of historical query requests. The intermediate layer may add, in the query cache, the query request received within the predetermined time range and the target data corresponding to the query request received within the predetermined time range, or add, in the query cache, the query request whose request frequency exceeds the predetermined frequency and the target data corresponding to the query request whose request frequency exceeds the predetermined frequency.
When the intermediate layer stores the query cache, after step 101, the method includes: the intermediate layer judges whether the query cache comprises the query request or not; if so, the intermediate layer acquires the target data corresponding to the query request in the query cache, and sends the target data acquired from the query cache to the client.
By the method, when the query request is one of the historical query requests, the intermediate layer can directly acquire the target data according to the query cache stored before, a metadata screening process and a process of reading data corresponding to the target metadata from the storage layer are not needed, and the data query efficiency is further improved.
Optionally, referring to fig. 1, the middle layer may include the following three functional modules: the device comprises a metadata storage module, a metadata updating module and a query cache module. The metadata storage module is used for acquiring and storing metadata of files in a storage layer, the metadata updating module is used for updating the metadata in the metadata storage module or updating the metadata in the storage layer according to metadata updating information sent by a client, and the updating mode can specifically comprise two modes of further supplementing original metadata and expanding new metadata; the query cache module is used for caching the historical query requests sent by the client and target metadata or target data corresponding to the historical query requests. Through the intermediate layer provided by the method, the technical effects of saving the development cost of the client, reducing the loading capacity of data query and the invalid calculation capacity of the client and improving the data query efficiency can be achieved.
Example two
The second embodiment of the present invention provides a data request method, including:
a client sends a query request to an intermediate layer, wherein the query request comprises query conditions and a storage format of target data, the query conditions are used for defining constraint conditions of metadata of the target data so as to enable the intermediate layer to obtain the target metadata meeting the query conditions, a target read command for reading the target data is determined according to the mapping between the storage format of the data and the read command, the data corresponding to the target metadata is read from a storage layer according to the determined target read command, and the data corresponding to the target metadata is sent to the client; wherein the data corresponding to the target metadata comprises the target data;
and the client receives the data corresponding to the target metadata sent by the middle layer.
Optionally, the sending, by the client, the query request to the middle layer includes:
and the client sends the query request to the client according to a preset protocol, wherein the preset protocol is a protocol which is followed by different clients and the intermediate layer for communication.
Optionally, after the client receives the data corresponding to the target metadata sent by the middle layer, the method further includes:
the client carries out statistical analysis on data corresponding to the target metadata to generate metadata updating information;
and sending the metadata updating information to the middle layer so that the middle layer updates the metadata stored in the middle layer according to the metadata updating information and/or updates the metadata in the storage layer according to the metadata updating information.
EXAMPLE III
A third embodiment of the present invention provides a data query device, configured to implement the data query method in the first embodiment, with reference to fig. 4, where the data query device includes:
a receiving unit 201, configured to receive an inquiry request sent by a client, where the inquiry request includes an inquiry condition and a storage format of target data; wherein the query condition is used for defining a constraint condition of metadata of the target data;
a determining unit 202, configured to obtain target metadata that meets the query condition, and determine a target read command for reading the target data according to a mapping between a storage format of data and the read command;
a reading unit 203, configured to read data corresponding to the target metadata from a storage layer according to the determined target read command;
a sending unit 204, configured to send data corresponding to the target metadata to the client, where the data corresponding to the target metadata includes the target data.
Optionally, the determining unit 202 is configured to: acquiring metadata used for indicating a file where the target data is located; or acquiring metadata used for indicating a row group where the target data is located; or acquiring metadata used for indicating the page where the target data is located.
Optionally, the determining unit 202 is configured to: acquiring target metadata meeting the query condition from the storage layer; and/or obtaining target metadata satisfying the query condition from metadata stored by the device.
Optionally, the receiving unit 201 is further configured to: receiving metadata updating information sent by the client; the metadata updating information is generated by performing statistical analysis on data corresponding to the target metadata after the client receives the data corresponding to the target metadata from the device;
the apparatus further comprises an update unit configured to: updating the metadata stored by the device according to the metadata updating information, and/or updating the metadata in the storage layer according to the metadata updating information.
Optionally, the type of the target metadata includes at least one of a numeric type, a time type, a character type, and an enumeration type.
Optionally, the apparatus stores a query cache, where the query cache includes at least one item of historical query request and target metadata corresponding to each item in the at least one item of historical query request;
the determining unit 202 is configured to: and judging whether the query cache comprises the query request or not, and acquiring target metadata corresponding to the query request in the query cache when the query cache comprises the query request.
Optionally, the apparatus further includes a cache unit, configured to: adding a query request received in a preset time range and target metadata corresponding to the query request received in the preset time range in the query cache; and/or adding query requests with the request times exceeding the preset times and target metadata corresponding to the query requests with the request times exceeding the preset times in the query cache.
Optionally, the receiving unit 201 is configured to: and receiving the query request sent by the client according to a predetermined protocol, wherein the predetermined protocol is a protocol which is followed by different clients and the device to communicate.
The specific implementation manner of the operations performed by the units may be the corresponding steps performed by the middle layer in the first embodiment of the present invention, and details of the embodiments of the present invention are not described again.
Example four
A fourth embodiment of the present invention provides a client, configured to implement the data request method in the second embodiment, with reference to fig. 5, where the client includes:
a sending unit 301, configured to send an inquiry request to an intermediate layer, where the inquiry request includes an inquiry condition and a storage format of target data, where the inquiry condition is used to define a constraint condition of metadata of the target data, so that the intermediate layer obtains the target metadata that meets the inquiry condition, determines a target read command for reading the target data according to a mapping between the storage format of the data and the read command, reads data corresponding to the target metadata from a storage layer according to the determined target read command, and sends the data corresponding to the target metadata to the client; wherein the data corresponding to the target metadata comprises the target data;
a receiving unit 302, configured to receive data corresponding to the target metadata sent by the middle layer.
Optionally, the client further includes a processing unit, configured to: after the receiving unit 302 receives the data corresponding to the target metadata sent by the middle layer, performing statistical analysis on the data corresponding to the target metadata to generate metadata update information;
the sending unit 301 is further configured to: and sending the metadata updating information to the middle layer so that the middle layer updates the metadata stored in the middle layer according to the metadata updating information and/or updates the metadata in the storage layer according to the metadata updating information.
Optionally, the sending unit 301 is configured to: and sending the query request to the middle layer according to a preset protocol, wherein the preset protocol is a protocol which is followed by different clients and the middle layer to communicate.
The specific implementation manner of the operations executed by the above units may be the corresponding steps executed by the client in the second embodiment of the present invention, and the embodiments of the present invention are not described again.
EXAMPLE five
An embodiment of the present invention further provides a data query device, including:
at least one processor, and
a memory communicatively coupled to the at least one processor, a communication interface;
the memory stores instructions executable by the at least one processor, and the at least one processor executes the data query method according to the first embodiment of the present invention by using the communication interface by executing the instructions stored in the memory.
EXAMPLE six
An embodiment of the present invention provides a client, including:
at least one processor, and
a memory communicatively coupled to the at least one processor, a communication interface;
the memory stores instructions executable by the at least one processor, and the at least one processor executes the method for requesting data according to the second embodiment of the present invention by using the communication interface by executing the instructions stored in the memory.
One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:
1. the access of the client to the data in the storage layer is indirectly realized through the middle layer, which is different from the direct interaction mode of the client and the storage layer in the prior art, the embodiment of the invention does not need to switch the types of program codes for the client when inquiring the target data in different storage formats in the storage layer, and does not need to develop a set of adaptive program codes for the client aiming at each storage format in the stage of developing the client, thereby reducing the development cost of the client;
2. after receiving a query request sent by a client, the intermediate layer can directly acquire metadata of a file to be queried from the metadata stored in the intermediate layer, and target metadata can be acquired without reading the file in the storage layer, so that the data loading amount can be effectively reduced, and the data query efficiency is improved;
3. the middle layer can receive metadata updating information from the client, continuously updates original metadata, and continuously expands and supplements the metadata, so that the metadata can describe more detailed data, and further the middle layer can acquire more accurate data when inquiring target data from the storage layer, the more accurate data can be obtained to a page level, the loading capacity of the data is further reduced, and the data inquiring efficiency is improved;
4. when the query request is one of historical query requests in the query cache, the intermediate layer can directly determine target metadata according to the query cache stored before, a metadata screening process is not needed, and the data query efficiency is further improved;
5. when the query request is one of the historical query requests in the query cache, the intermediate layer can directly acquire the target data according to the query cache stored before, and a metadata screening process and a process of reading data corresponding to the target metadata from the storage layer are not needed, so that the data query efficiency is further improved.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (22)
1. A method for querying data, comprising:
the intermediate layer receives a query request sent by a client, wherein the query request comprises a query condition and a storage format of target data; wherein the query condition is used for defining a constraint condition of metadata of the target data;
the intermediate layer obtains target metadata satisfying the query condition, an
Determining a target read command for reading the target data according to the mapping between the storage format of the data and the read command;
and the middle layer reads data corresponding to the target metadata from a storage layer according to the determined target reading command and sends the data corresponding to the target metadata to the client, wherein the data corresponding to the target metadata comprises the target data.
2. The method of claim 1, wherein the middle tier obtaining target metadata that satisfies the query condition comprises:
the middle layer acquires metadata used for indicating a file where the target data is located; or
The middle layer acquires metadata used for indicating a row group where the target data is located; or
And the middle layer acquires metadata used for indicating the page where the target data is located.
3. The method of claim 1, wherein the middle tier obtaining target metadata that satisfies the query condition comprises:
the middle layer acquires target metadata meeting the query condition from the storage layer; and/or
And the middle layer acquires target metadata meeting the query condition from the metadata stored in the middle layer.
4. The method of claim 1, further comprising:
the middle layer receives metadata updating information sent by the client; the metadata updating information is generated by performing statistical analysis on data corresponding to the target metadata after the client receives the data corresponding to the target metadata from the middle layer;
the middle layer updates the metadata stored in the middle layer according to the metadata updating information; and/or
And the middle layer updates the metadata in the storage layer according to the metadata updating information.
5. The method of claim 1, wherein the type of the target metadata comprises at least one of a numeric type, a temporal type, a character type, an enumeration type.
6. The method of claim 1, wherein the middle layer maintains a query cache, the query cache including at least one historical query request and target metadata corresponding to each of the at least one historical query request;
the intermediate layer acquires target metadata meeting the query condition, and the method comprises the following steps:
the intermediate layer judges whether the query cache comprises the query request or not;
if so, the intermediate layer acquires the target metadata corresponding to the query request in the query cache.
7. The method of claim 6, further comprising:
the intermediate layer adds a query request received in a preset time range and target metadata corresponding to the query request received in the preset time range in the query cache; and/or
And the intermediate layer adds the query requests with the request times exceeding the preset times and the target metadata corresponding to the query requests with the request times exceeding the preset times in the query cache.
8. The method of claim 1, wherein the receiving, by the middle tier, the query request sent by the client comprises:
and the intermediate layer receives the query request sent by the client according to a preset protocol, wherein the preset protocol is a protocol which is followed by different clients and the intermediate layer when the different clients communicate.
9. A method of requesting data, comprising:
a client sends a query request to an intermediate layer, wherein the query request comprises query conditions and a storage format of target data, the query conditions are used for defining constraint conditions of metadata of the target data so as to enable the intermediate layer to obtain the target metadata meeting the query conditions, a target read command for reading the target data is determined according to the mapping between the storage format of the data and the read command, the data corresponding to the target metadata is read from a storage layer according to the determined target read command, and the data corresponding to the target metadata is sent to the client; wherein the data corresponding to the target metadata comprises the target data;
and the client receives the data corresponding to the target metadata sent by the middle layer.
10. The method of claim 9, wherein after the client receives the data corresponding to the target metadata sent by the middle tier, the method further comprises:
the client carries out statistical analysis on data corresponding to the target metadata to generate metadata updating information; and sending the metadata updating information to the middle layer so that the middle layer updates the metadata stored in the middle layer according to the metadata updating information and/or updates the metadata in the storage layer according to the metadata updating information.
11. The method of claim 9, wherein the client sends a query request to the middle tier, comprising:
and the client sends the query request to the intermediate layer according to a preset protocol, wherein the preset protocol is a protocol which is followed by different clients and the intermediate layer when the clients communicate with the intermediate layer.
12. A data query apparatus, comprising:
the system comprises a receiving unit, a sending unit and a processing unit, wherein the receiving unit is used for receiving a query request sent by a client, and the query request comprises a query condition and a storage format of target data; wherein the query condition is used for defining a constraint condition of metadata of the target data;
the determining unit is used for acquiring target metadata meeting the query condition and determining a target read command for reading the target data according to the mapping between the storage format of the data and the read command;
the reading unit is used for reading data corresponding to the target metadata from the storage layer according to the determined target reading command;
and the sending unit is used for sending the data corresponding to the target metadata to the client, and the data corresponding to the target metadata comprises the target data.
13. The apparatus of claim 12, wherein the determination unit is to:
acquiring metadata used for indicating a file where the target data is located; or
Acquiring metadata used for indicating a row group where the target data is located; or
And acquiring metadata used for indicating the page where the target data is located.
14. The apparatus of claim 12, wherein the determination unit is to:
acquiring target metadata meeting the query condition from the storage layer; and/or
And acquiring target metadata meeting the query condition from the metadata stored in the device.
15. The apparatus of claim 12, wherein the receiving unit is further configured to:
receiving metadata updating information sent by the client; the metadata updating information is generated by performing statistical analysis on data corresponding to the target metadata after the client receives the data corresponding to the target metadata from the device;
the apparatus further comprises an update unit configured to: updating the metadata stored by the device according to the metadata updating information, and/or updating the metadata in the storage layer according to the metadata updating information.
16. The apparatus of claim 12, wherein the type of the target metadata comprises at least one of a numeric type, a temporal type, a character type, an enumeration type.
17. The apparatus of claim 12, wherein the apparatus maintains a query cache, the query cache including at least one historical query request and target metadata corresponding to each of the at least one historical query request;
the determination unit is configured to: and judging whether the query cache comprises the query request or not, and acquiring target metadata corresponding to the query request in the query cache when the query cache comprises the query request.
18. The apparatus of claim 17, further comprising a caching unit to:
adding a query request received in a preset time range and target metadata corresponding to the query request received in the preset time range in the query cache; and/or
And adding query requests with the request times exceeding the preset times and target metadata corresponding to the query requests with the request times exceeding the preset times in the query cache.
19. The apparatus of claim 12, wherein the receiving unit is to:
and receiving the query request sent by the client according to a predetermined protocol, wherein the predetermined protocol is a protocol which is followed by different clients and the device to communicate.
20. A client, comprising:
a sending unit, configured to send a query request to an intermediate layer, where the query request includes a query condition and a storage format of target data, where the query condition is used to define a constraint condition of metadata of the target data, so that the intermediate layer obtains the target metadata that meets the query condition, determines a target read command for reading the target data according to a mapping between the storage format of the data and the read command, reads data corresponding to the target metadata from a storage layer according to the determined target read command, and sends the data corresponding to the target metadata to the client; wherein the data corresponding to the target metadata comprises the target data;
and the receiving unit is used for receiving the data corresponding to the target metadata sent by the middle layer.
21. The client of claim 20, wherein the client further comprises:
the processing unit is used for performing statistical analysis on the data corresponding to the target metadata after the receiving unit receives the data corresponding to the target metadata sent by the middle layer, and generating metadata updating information;
the sending unit is further configured to: and sending the metadata updating information to the middle layer so that the middle layer updates the metadata stored in the middle layer according to the metadata updating information and/or updates the metadata in the storage layer according to the metadata updating information.
22. The client of claim 20, wherein the sending unit is further configured to:
and sending the query request to the middle layer according to a preset protocol, wherein the preset protocol is a protocol which is followed by different clients and the middle layer to communicate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710801101.2A CN107679093B (en) | 2017-09-07 | 2017-09-07 | Data query method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710801101.2A CN107679093B (en) | 2017-09-07 | 2017-09-07 | Data query method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107679093A CN107679093A (en) | 2018-02-09 |
| CN107679093B true CN107679093B (en) | 2020-04-07 |
Family
ID=61135052
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710801101.2A Active CN107679093B (en) | 2017-09-07 | 2017-09-07 | Data query method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107679093B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110059105A (en) * | 2019-04-26 | 2019-07-26 | 北京贝斯平云科技有限公司 | A kind of data processing method, device, computer equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101980213A (en) * | 2010-11-23 | 2011-02-23 | 中国科学院软件研究所 | J2EE-based data persistence method and system |
| CN105005522A (en) * | 2015-06-30 | 2015-10-28 | 深圳市芯海科技有限公司 | Online simulation method of LCD nixie tube |
| CN105224546A (en) * | 2014-06-04 | 2016-01-06 | 阿里巴巴集团控股有限公司 | Data storage and query method and apparatus |
| US9256697B2 (en) * | 2012-05-11 | 2016-02-09 | Microsoft Technology Licensing, Llc | Bidirectional mapping between applications and network content |
| CN105975493A (en) * | 2016-04-26 | 2016-09-28 | 北京神州绿盟信息安全科技股份有限公司 | File combination method and apparatus |
| CN106776733A (en) * | 2016-11-22 | 2017-05-31 | 北京航天自动控制研究所 | The data staging methods of sampling in data handling system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090024596A1 (en) * | 2007-07-18 | 2009-01-22 | At&T Labs, Inc. | System and apparatus to represent, store, manipulate, and process metadata information |
-
2017
- 2017-09-07 CN CN201710801101.2A patent/CN107679093B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101980213A (en) * | 2010-11-23 | 2011-02-23 | 中国科学院软件研究所 | J2EE-based data persistence method and system |
| US9256697B2 (en) * | 2012-05-11 | 2016-02-09 | Microsoft Technology Licensing, Llc | Bidirectional mapping between applications and network content |
| CN105224546A (en) * | 2014-06-04 | 2016-01-06 | 阿里巴巴集团控股有限公司 | Data storage and query method and apparatus |
| CN105005522A (en) * | 2015-06-30 | 2015-10-28 | 深圳市芯海科技有限公司 | Online simulation method of LCD nixie tube |
| CN105975493A (en) * | 2016-04-26 | 2016-09-28 | 北京神州绿盟信息安全科技股份有限公司 | File combination method and apparatus |
| CN106776733A (en) * | 2016-11-22 | 2017-05-31 | 北京航天自动控制研究所 | The data staging methods of sampling in data handling system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107679093A (en) | 2018-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104462225B (en) | The method, apparatus and system of a kind of digital independent | |
| CN110046133B (en) | Metadata management method, device and system for storage file system | |
| CN110909025A (en) | Database query method, query device and terminal | |
| CN110147407B (en) | Data processing method and device and database management server | |
| US9619503B2 (en) | Method, server, and system for accessing metadata | |
| CN103019960A (en) | Distributed cache method and system | |
| CN112039979A (en) | Distributed data cache management method, device, equipment and storage medium | |
| CN110134738B (en) | Distributed storage system resource estimation method and device | |
| CN110851474A (en) | Data query method, database middleware, data query device and storage medium | |
| CN111737564A (en) | Information query method, device, equipment and medium | |
| CN113157609B (en) | Storage system, data processing method, data processing device, electronic equipment and storage medium | |
| EP2621143A1 (en) | Information processing apparatus, distributed processing system, and distributed processing method | |
| CN111651424B (en) | Data processing method, device, data node and storage medium | |
| CN111291083B (en) | Webpage source code data processing method and device and computer equipment | |
| CN110222046B (en) | List data processing method, device, server and storage medium | |
| CN101247405A (en) | Method, system and device for calculating download time and resource downloading | |
| CN111125156A (en) | Data query method and device and electronic equipment | |
| CN113064919B (en) | Data processing method, data storage system, computer device and storage medium | |
| CN107679093B (en) | Data query method and device | |
| CN111639087B (en) | Data updating method and device in database and electronic equipment | |
| CN111831691A (en) | Data reading and writing method and device, electronic equipment and storage medium | |
| CN113626449A (en) | Data storage method, data query method and related equipment | |
| CN112486988A (en) | Data processing method, device, equipment and storage medium | |
| CN110765125B (en) | Method and device for storing data | |
| CN117009389A (en) | Data caching method, device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100089 Beijing city Haidian District Road No. 4 North wa Yitai three storey building Patentee after: NSFOCUS Technologies Group Co.,Ltd. Patentee after: NSFOCUS TECHNOLOGIES Inc. Address before: 100089 Beijing city Haidian District Road No. 4 North wa Yitai three storey building Patentee before: NSFOCUS INFORMATION TECHNOLOGY Co.,Ltd. Patentee before: NSFOCUS TECHNOLOGIES Inc. |