CN115563235A

CN115563235A - Hotspot-aware log structure merged tree read-write performance optimization method and related equipment

Info

Publication number: CN115563235A
Application number: CN202211294729.5A
Authority: CN
Inventors: 王芳; 冯丹; 张健顺; 董超
Original assignee: Huazhong University of Science and Technology
Current assignee: Huazhong University of Science and Technology
Priority date: 2022-10-21
Filing date: 2022-10-21
Publication date: 2023-01-03

Abstract

The invention discloses a method for optimizing read-write performance of a hotspot-aware log structure merged tree and related equipment, belonging to the technical field of data storage and comprising the following steps: maintaining a coarse-grained cache and a fine-grained cache in a memory, and caching recently accessed data in the system by taking a data block and a key value pair as granularities respectively; the coarse-grained cache is read-only cache and is used for service range query operation; the fine-grained cache is read-write mixed cache and is used for query operation and write operation of a service point; a data merging method for hot spot perception is introduced to a hard disk, a calculation method for realizing data heat is designed, execution of internal data merging operation in a log structure merging tree is scheduled based on data access heat, meanwhile, invalid data participating in merging are pruned quickly, and hot new data blocks are prefetched to a coarse-grained cache after merging is completed. The method and the device can improve the cache hit rate, shorten the read-write path and optimize the read-write performance of the log structure merging tree.

Description

Hotspot-aware log structure merged tree read-write performance optimization method and related equipment

Technical Field

The invention belongs to the technical field of data storage, and particularly relates to a hotspot-aware log structure merged tree read-write performance optimization method and related equipment.

Background

With the rapid development of information technology and the explosive increase of data volume, mass data are generated in various industry fields, the performance requirement of a storage system is further improved, a relatively fixed data model is often used in the traditional data storage system to organize and manage data, and the data model is usually established based on some simple data scenes, for example, a simple logic two-dimensional table is used to express and display data, so that the unstructured data storage system cannot meet the trend of diversification of data forms and the concurrent read-write requirement of high performance in the big data era.

Unstructured databases based on Log-Structured Merge Trees (LSM-Trees) have become the infrastructure of modern storage systems in recent years, and key data storage services are provided for the outside in the form of key value storage. The storage system based on the log structure merging tree fully considers the characteristic that the sequential read-write performance of the storage device is far better than the random read-write performance, obtains higher write performance by adopting a memory delayed batch write mode, and is widely applied to the write-intensive application scene.

Although the log-structured merge tree can provide excellent writing performance, the read performance of the log-structured merge tree is lower than the write performance because the merge operation needs to be continuously performed in the background to delete the expired data and ensure the orderliness of the data on the storage device in time, and thus the log-structured merge tree is not enough to be applied to a wider read-write mixed scene. Secondly, the current application load often contains data hotspots, that is, a small part of data occupies most of request access, but the existing log structure merge tree does not consider the hotspot characteristics of the load, and also causes the hotspot data to be repeatedly read and written on the storage device due to the merge operation, and contend for the internal server resource with the user request of the foreground, which causes the reduction of the read-write performance.

In general, in the existing storage system based on the log structure merge tree, the read-write performance needs to be further improved.

Disclosure of Invention

Aiming at the defects and improvement requirements of the prior art, the invention provides a hotspot-aware method and related equipment for optimizing the read-write performance of a log structure merging tree, and aims to optimize the read-write performance of the log structure merging tree.

To achieve the above object, according to an aspect of the present invention, a method for optimizing read and write performance of a hotspot-aware log-structured merge tree is provided, including:

maintaining a coarse-grained cache and a fine-grained cache in a memory, and caching recently accessed data in the system by taking a data block and a key value pair as granularities respectively;

and, a point query operation for querying the target key-value pair, the execution of which comprises:

(R1) inquiring a target key value pair in a fine-grained cache, and if the inquiry is successful, turning to the step (R4); otherwise, turning to the step (R2);

(R2) sequentially accessing each component according to the sequence of write buffering, read-only write buffering, coarse-grained caching and log structure merging trees until a target key value pair is inquired, and if the target key value pair is not inquired in all the components, finishing the point inquiry operation; otherwise, turning to the step (R3);

(R3) caching the inquired key value pair into a fine-grained cache; if the target key value pair is inquired in the log structure merging tree, caching the data block where the target key value pair is in into a coarse-grained cache;

and (R4) returning the inquired key value pair, and ending the point inquiry operation.

Furthermore, a dirty data queue is maintained in the fine-grained cache and used for recording dirty data generated by updating in the fine-grained cache;

and, the performing of the write operation includes:

(W1) writing a key value pair to be written into a log before writing;

(W2) judging whether the key value pair to be written is located in the fine-grained cache, if so, writing the key value pair to be written into the fine-grained cache in a local updating mode, recording the data to be written into a dirty data queue, setting a corresponding dirty data identification bit, and then turning to the step (W3); otherwise, writing the key value pair to be written into the write buffer, and then turning to the step (W3);

(W3) returning the updating result and ending the writing operation.

Further, a scope query operation for querying key-value pairs within a specified scope, the execution of which comprises:

(S1) if the dirty data queue is not empty, reading and sequencing all the dirty data in the fine-grained cache in batches, writing the dirty data into a write buffer in batches, and then turning to the step (S2); otherwise, directly switching to the step (S2);

(S2) reading key value pairs in a specified range from a write buffer and a read-only write buffer, determining data blocks which overlap with the specified range in the daily combination structure merging tree and have the key value range, if the data blocks are cached to a coarse-grained cache, reading corresponding data blocks from the coarse-grained cache, and if not, reading the data blocks from the log structure merging tree and caching the data blocks into the coarse-grained cache; reading key-value pairs located within a specified range from the read data blocks;

and (S4) merging the read key value pairs, returning a merged key value pair set, and finishing the range query operation.

Further, the coarse-grained cache is divided into a plurality of first partitions, and a hash table is adopted in each first partition to manage the data blocks; the fine-grained cache is divided into a plurality of second partitions, and a hash table is adopted in each second partition to manage the key value pairs.

Further, the merging operation of the log-structured merge tree includes:

determining a hierarchy needing to initiate merging in a log structure merging tree, calculating the heat of each data table in the hierarchy, and selecting the data table with the minimum heat to initiate merging operation;

wherein, the heat of the data table is the sum of the heat of all the data blocks in the data table; the heat of the data block is the sum of the heat of all key-value pairs in the data block.

Further, in the log-structured merged tree, the heat calculation mode of the key-value pairs includes:

judging whether the key value pair is in the fine-grained cache, if so, setting the read heat degree to be s _r Otherwise, setting the reading heat degree to 0; s _r >0；

Judging whether the key-value pair is discarded in the previous merging operation, if so, setting the write heat degree of the key-value pair as s _w Otherwise, setting the writing heat degree to 0; s _w >0；

And taking the sum of the read heat and the write heat of the key-value pair as the heat of the key-value pair.

Further, the merging operation of the log-structured merge tree further includes:

and for new data blocks generated in the merging process, calculating the heat degree and the average heat degree of each data block, selecting the data blocks with the heat degrees higher than the average heat degree, and replacing the data blocks which fail due to merging in the coarse-grained cache.

and for the key value pair needing to be merged in the log-structured merge tree, if the data of the updated version is cached in the fine-grained cache, the write buffer or the read-only write buffer, discarding the key value pair.

According to another aspect of the present invention, there is provided a memory controller including: a computer-readable storage medium and a processor; a computer program is stored in a computer readable storage medium; the processor is used for reading a computer program in a computer readable storage medium and executing the method for optimizing the read-write performance of the hotspot-aware log structure merging tree provided by the invention.

According to still another aspect of the present invention, there is provided a log-structured merge-tree based storage system, including: the invention provides a memory, a memory device and the memory controller.

Generally, by the above technical solution conceived by the present invention, the following beneficial effects can be obtained:

(1) The method and the related equipment for optimizing the read-write performance of the hotspot-aware log structure merging tree maintain double-granularity cache, namely coarse-granularity cache and fine-granularity cache, which respectively use data blocks and key value pairs as granularity for data cache, wherein the fine-granularity cache is used for caching the recently accessed key value pair and is used as a first component of a service point query request.

(2) According to the method and the related device for optimizing the read-write performance of the hotspot-aware log-structured merge tree, in the preferred scheme, the fine-grained cache maintained in the memory is simultaneously used as a first component for servicing write operation, so that the hotspot characteristics of data load can be fully considered, the access path of the write operation is effectively shortened, and the execution performance of the write operation is improved.

(3) According to the hotspot-aware log structure merged tree read-write performance optimization method and the related equipment, in the preferred scheme, both the coarse-grained cache and the fine-grained cache perform data management in a way of combining partitions and hash tables, specifically, the cache is divided into a plurality of partitions, and each partition performs data management by using the hash table, so that the concurrent read-write capability of the cache can be effectively improved, and the read-write performance is further improved.

(4) In the preferred scheme, when the combining operation executed in the background selects the data table for initiating the combining, the heat degree of each data table in the current layer of the log structure combining tree is calculated firstly, and the data table with the minimum heat degree is selected as an initiator of the combining operation, so that the hotter data tables can be ensured to be kept at the upper layer of the log structure combining tree as much as possible, a writing request or a service reading request can be absorbed quickly, the reading and writing path of the log structure combining tree is shortened, and the reading and writing performance is further improved; further preferably, when calculating the hot degree of the key value pair in the data table, the hot and cold of the single key value pair data is identified by combining the fine-grained cache and the data discard record of the previous merging operation, and specifically, considering that in the present invention, the data in the fine-grained cache is only inserted into the fine-grained cache in the process of the point query operation, the present invention identifies the data in the fine-grained cache as hot read data, and if the key value pair data is discarded in the previous merging operation, it indicates that the data is written in multiple times, therefore, the present invention identifies the data discarded in the previous merging operation as hot write data, so that the hot spot perception capability of the log structure merging tree can be improved based on the management mechanism of the data, and the hot degree of the key value pair, the data block and the data table can be calculated accurately and efficiently.

(5) According to the hotspot-aware log structure merge tree read-write performance optimization method and the related device, in the preferred scheme, in the merge operation, a data block with high heat is selected from newly generated data blocks, and a data block which fails due to the merge operation in the coarse-grained cache is replaced, so that the prefetching of hot data is realized, the cache failure of the coarse-grained cache due to the merge operation can be reduced, and the cache hit rate is improved.

(6) According to the hotspot-aware log structure merging tree read-write performance optimization method and the related device, in the preferred scheme, the data which are invalid in the past period can be identified in the merging operation and are directly discarded, so that repeated read-write of the invalid data on the storage device can be reduced, read-write amplification and space amplification are reduced, the read-write performance is improved, and the service life of the storage device is prolonged.

Drawings

Fig. 1 is a schematic diagram of a method for optimizing read-write performance of a hotspot-aware log-structured merge tree according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a write operation provided by an embodiment of the present invention;

FIG. 3 is a schematic diagram of a merge operation according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a pruning mechanism in a merge operation according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

In the present application, the terms "first," "second," and the like (if any) in the description and the drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

In order to further improve the read-write performance of the log structure merging tree, the invention provides a method for optimizing the read-write performance of the log structure merging tree with hotspot perception and related equipment, and the overall thought is as follows: the sensing capability of the system to the load hot spot is improved, the hot data is kept at the upper layer of the read-write path as much as possible, the read-write path is shortened, the expense of the merging operation in the log structure merging tree is reduced, and the read-write performance is improved.

The following are examples.

Example 1:

a method for optimizing read-write performance of a hotspot-aware log-structured merge tree is disclosed, as shown in FIG. 1, and includes: maintaining a coarse-grained cache and a fine-grained cache in a memory, and caching recently accessed data in the system by taking a data block and a key value pair as granularities respectively;

in this embodiment, the coarse-grained cache is a read-only cache, and the data block is used as the granularity management data, so that the query operation in the service range can be performed; optionally, the size of the data block is 4KB; the fine-grained cache is read-write mixed cache, key value pairs (with different sizes) are used as grained management data, and query operation and write operation can be served.

In order to speed up reading and writing of data in the cache, as a preferred implementation manner, in this embodiment, both the coarse-grained cache and the fine-grained cache manage data in a manner of combining partitions with hash tables, specifically, the cache is divided into a plurality of partitions, and one hash table is established for each partition, so that concurrent reading and writing capability of the cache can be effectively improved, and the reading and writing performance is further improved; when accessing data, firstly determining a cache partition corresponding to the data, and then calculating a hash value by using a hash function corresponding to the partition, namely determining the storage position of the data in the partition. The specific partition size and hash function may be set according to actual load characteristics. Optionally, in this embodiment, when cache replacement occurs, both the coarse-grained cache and the fine-grained cache use a classic LRU algorithm to determine the replaced data, and accordingly, an LRU queue is maintained in the cache to quickly determine the obsolete data.

In the present embodiment, a fine-grained cache is used as a first component of a service point query request, and accordingly, as shown in fig. 1, in the present embodiment, a point query operation for querying a target key-value pair is performed, and includes:

(R2) sequentially accessing each component according to the sequence of the write buffer, the read-only write buffer, the coarse-grained cache and the log structure merge tree until a target key value pair is inquired, and if the target key value pair is not inquired in all the components, ending the point inquiry operation; otherwise, turning to the step (R3);

(R3) caching the inquired key-value pair into a fine-grained cache; if the target key value pair is inquired in the log structure merging tree, caching the data block where the target key value pair is in into a coarse-grained cache;

(R4) returning the inquired key value pair, and finishing the point inquiry operation;

in this embodiment, a fine-grained cache is used as a first component of a service point query request, and a dirty data queue is maintained in the fine-grained cache, and is used to record dirty data generated due to updating in the fine-grained cache, and accordingly, as shown in fig. 2, in this embodiment, the performing of the write operation includes:

(W1) writing the key value pair to be written into the log before writing;

(W2) judging whether the key value pair to be written is located in the fine-grained cache, if so, writing the key value pair to be written into the fine-grained cache in a local updating mode, recording the data to be written into a dirty data queue, setting a corresponding dirty data flag bit, and then turning to the step (W3); otherwise, writing the key value pair to be written into the write buffer, and then turning to the step (W3);

(W3) returning an updating result, and ending the writing operation;

for a data write request hit in the fine-grained cache, recording the generated dirty data in a dirty data queue, and setting a dirty data identification bit corresponding to the data; when the LRU elimination algorithm is executed, the state of the data to be evicted is judged according to the dirty data flag bit corresponding to the data, if the state indicates that the data is dirty data, the data is correspondingly inserted into a write buffer, so that the data can be correctly persisted on a storage device finally, and if the data is clean read-only data, the data is directly discarded to save the buffer space.

The steps (W1) to (W3) complete foreground processing of the write operation, and in addition to the steps (W1) to (W3), further processing is performed through a background thread to ensure that data is persisted to the storage device; the background processing of the write operation comprises the following steps:

after the write buffer is full, changing the state of the write buffer into a read-only state, converting the write buffer into a read-only write buffer, and simultaneously creating a new write buffer to continuously receive data write;

and (3) flushing the data in the read-only write buffer back to the persistent storage device in the form of a data table (namely SSTable) to realize data landing.

In this embodiment, the range query operation for querying key-value pairs within a specified range is performed by:

(S1) if the dirty data queue is not empty, all dirty data in the fine-grained cache are read in batch and sequenced, then written into a write buffer in batch, and then the step (S2) is carried out; otherwise, directly switching to the step (S2);

because the fine-grained cache takes key value pairs as the granularity management data, the data in the fine-grained cache is unordered and cannot support range query operation; in this embodiment, before performing the range query operation, dirty data in the fine-grained cache is written into the write buffer, so that it is ensured that data queried by the range query operation are all latest data;

(S2) reading key value pairs in a specified range from a write buffer and a read-only write buffer, determining data blocks which overlap with the key value range in the specified range in a combined tree of a daily structure, if the data blocks are cached in a coarse-grained cache, reading corresponding data blocks from the coarse-grained cache, and if not, reading the data blocks from the combined tree of the log structure and caching the data blocks into the coarse-grained cache; reading key-value pairs located within a specified range from the read data blocks;

in this embodiment, when reading a data block from a log-structured merge tree, it is first determined whether the data block is located in a coarse-grained cache, and if so, the data block is directly read from the coarse-grained cache; therefore, the read-write path can be further shortened by means of coarse-grained cache, and the read-write performance of the log structure merging tree is improved;

(S4) merging the read key value pairs, returning a merged key value pair set, and finishing the range query operation;

during the merge process, invalid data is discarded and the retained data is sorted.

Based on coarse-grained cache and fine-grained cache, the embodiment realizes double-grained cache, and the embodiment takes the fine-grained cache as a first component for serving the read-write request, can fully utilize the hot spot characteristics of the load and reserve hot spot data in the upper-layer components as much as possible, thereby effectively shortening the read-write path, reducing the merging operation on the log structure merging tree and improving the read-write performance; on the basis, the cache is partitioned, and the memory hash table structure is used for accelerating the reading and writing of data, so that the reading and writing performance can be further improved. The coarse-granularity cache and the fine-granularity cache cooperate to provide services for read requests with different granularities, the coarse-granularity cache makes up the disadvantage that the fine-granularity cache cannot efficiently process range query, and the fine-granularity cache makes up the disadvantage that the coarse-granularity cache has lower space utilization rate. The double-granularity cache has the combined action and is suitable for various loads.

In order to further improve the read-write performance of the log-structured merge tree, a hot spot identification mechanism is introduced into the merge operation executed in the background in this embodiment, and accordingly, the execution of the merge operation includes:

wherein, the heat of the data table is the sum of the heat of all the data blocks in the data table; the heat of the data block is the sum of the heat of all key value pairs in the data block;

in this embodiment, the hot calculation of the key value pair fully considers the hot characteristics of the load, and in view of that, in the present invention, data in the fine-grained cache is only inserted into the fine-grained cache in the process of the point query operation, so the key value pair located in the fine-grained cache often has a higher read hot degree, and if the key value pair data is discarded in the previous merge operation, it indicates that the data is written multiple times and therefore has a higher write hot degree, based on this, in the embodiment, when performing hot spot identification, the data cached in the fine-grained cache in the log structure merge tree is identified as hot read data, and the data discarded in the previous merge operation is identified as hot write data, and the calculation mode of the hot degree of the key value pair specifically includes:

judging whether the key value pair is in the fine-grained cache, if so, indicating that the data is hot read data, and setting the read hot degree to s _r Otherwise, setting the reading heat degree to 0; alternatively, in this embodiment, s _r ＝1；

Judging whether the key value pair is discarded in the previous merging operation, if so, indicating that the data is hot-written data, and setting the writing heat degree of the data as s _w Otherwise, setting the writing heat degree to 0; alternatively, in this embodiment, s _w ＝1；

Based on the above operation, the present embodiment implements a hot spot identification mechanism in the merge operation, and based on the hot spot identification mechanism, the present embodiment selects the data table with the minimum heat as an initiator of the merge operation, thereby ensuring that the hotter data table is kept at the upper layer of the log-structured merge tree as much as possible, so as to quickly absorb the write request or service the read request.

In the merging operation execution process of the log-structured merge tree, a new data block may be generated, and at the same time, a data block that has been cached in the coarse-grained cache may be invalidated, and in order to improve the cache hit rate, in this embodiment, the execution of the merging operation further includes:

calculating the heat and average heat of each data block for new data blocks generated in the merging process, selecting the data blocks with the heat greater than the average heat, and replacing the data blocks which fail due to merging in the coarse-grained cache;

based on the above operations, the embodiment implements a mechanism for prefetching the hot data block during the merge operation, thereby reducing cache invalidation of the coarse-grained cache due to the merge operation and increasing the cache hit rate.

The hot spot identification mechanism and the prefetching mechanism can be described according to the example shown in fig. 3, the heat of each data Table in the current layer L1 that needs to be merged is recorded in the memory by using a heat Table, where "Table" represents a data Table, and "Hotness" represents the heat of the corresponding data Table; based on the hot spot identification mechanism, selecting the coldest data table, namely T3, as the initiator of the merging operation, loading the data tables with overlapping ranges with the data table T3 in the next layer L2, namely T6 and T7 in the figure 3, into the memory together in the merging process, and executing the merging operation; merging operation generates new data tables, namely T8 and T9 in FIG. 3, the original data tables participating in merging are deleted inertly, the heat of each data block (B1-B4 in FIG. 3) in each newly generated data table is calculated simultaneously in the merging process, and the average heat is calculated; meanwhile, some data blocks in the coarse-grained cache may fail due to merge operation, and based on the above prefetching mechanism, data blocks with a heat degree greater than the average heat degree (B2 and B3 in fig. 3) are selected, prefetched into the coarse-grained cache, and replaced with the data blocks that failed due to merge before.

In this embodiment, a pruning mechanism is further designed to delete the expired invalid data in time, and avoid subsequent repeated reading and writing of the invalid data, so as to reduce the reading and writing amplification and optimize the reading and writing performance, and accordingly, the merging operation further includes:

for the key value pair needing to be merged in the log structure merging tree, if the data of the updated version is cached in a fine-grained cache, a write buffer or a read-only write buffer, discarding the key value pair;

based on the above operations, this embodiment implements a pruning mechanism, which can be further described by the example shown in fig. 4, in the process of merging data, traversing each piece of key-value pair data to be merged according to the order of keys, and correspondingly checking the validity of the piece of key-value pair data, i.e., checking whether the piece of data hits in the fine-grained cache, the write buffer, and the read-only write buffer, and whether the version included in the corresponding fine-grained cache is greater than that of the currently merged data, if so, determining that the data is invalid data, i.e.,

data

19, 21, and 77 in fig. 4, and directly discarding the data in the merging process, thereby avoiding subsequent repeated reading and writing of the piece of data; otherwise, the data is identified as valid data, i.e., data 28 to 56 and 81 in fig. 4, which cannot be deleted;

and after the data pruning is finished, writing the residual valid data back to the storage device to generate a new data table.

As a preferred implementation manner, in this embodiment, the identification of the expired data is completed in the hot spot data identification process, that is, in the data hot and cold identification process, whether the data is hot data is determined while the validity of the data is checked, and if the data is found to be invalid, the data is deleted in time from the merging process, so as to avoid subsequent repeated reading and writing of the invalid data. The invention combines the judgment of data validity and the hot spot identification of the data together, reduces the extra expense caused by the calculation of the data heat degree, and greatly reduces the expense generated in the merging process of invalid data because of the timely deletion of the invalid data, thereby further reducing the read-write amplification and the space amplification in the merging process, correspondingly reducing the abrasion of the storage equipment and prolonging the service life of the storage equipment.

Example 2:

a storage controller, comprising: a computer-readable storage medium and a processor; a computer program is stored in a computer readable storage medium; the processor is configured to read a computer program in a computer-readable storage medium, and execute the hotspot-aware log-structured merge-tree read-write performance optimization method provided in embodiment 1.

Example 3:

according to still another aspect of the present invention, there is provided a log-structured merge-tree-based storage system, including: a memory, a storage device, and the storage controller provided in embodiment 2 above.

It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A method for optimizing the read-write performance of a hotspot-aware log-structured merge tree is characterized by comprising the following steps:

(R1) inquiring the target key-value pair in the fine-grained cache, and if the inquiry is successful, turning to the step (R4); otherwise, turning to the step (R2);

(R2) sequentially accessing each component according to the sequence of write buffering, read-only write buffering, coarse-grained caching and the log structure merge tree until the target key value pair is inquired, and if the target key value pair is not inquired in all the components, finishing the point inquiry operation; otherwise, turning to the step (R3);

(R3) caching the queried key-value pair in the fine-grained cache; if the target key-value pair is inquired in the log structure merged tree, caching a data block where the target key-value pair is located in the coarse-grained cache;

and (R4) returning the inquired key value pair, and finishing the point inquiry operation.

2. The hotspot-aware log-structured merge-tree read-write performance optimization method of claim 1, wherein a dirty data queue is maintained in the fine-grained cache and used for recording dirty data generated by updating in the fine-grained cache;

and, the performing of the write operation includes:

(W1) writing a key value pair to be written into a log before writing;

(W2) judging whether the key value pair to be written is located in the fine-grained cache, if so, writing the key value pair to be written into the fine-grained cache in a local updating mode, recording the data to be written into the dirty data queue, setting a corresponding dirty data flag bit, and then turning to the step (W3); otherwise, writing the key value pair to be written into the write buffer, and then turning to the step (W3);

(W3) returning the updating result and ending the writing operation.

3. The hotspot-aware log-structured merge-tree read-write performance optimization method of claim 2, wherein a range query operation for querying key-value pairs within a specified range is performed comprising:

(S1) if the dirty data queue is not empty, reading and sequencing all dirty data in the fine-grained cache in batches, writing the dirty data into the write buffer in batches, and then turning to the step (S2); otherwise, directly switching to the step (S2);

(S2) reading key value pairs in the specified range from the write buffer and the read-only write buffer, determining data blocks in the daily combination structure merged tree, which overlap with the specified range in the key value range, if the data blocks are cached in the coarse-granularity cache, reading corresponding data blocks from the coarse-granularity cache, and if not, reading the data blocks from the log structure merged tree and caching the data blocks in the coarse-granularity cache; reading key-value pairs located within the specified range from the read data blocks;

4. The hotspot-aware log-structured merge-tree read-write performance optimization method of any one of claims 1-3, wherein the coarse-grained cache is divided into a plurality of first partitions, and a hash table is used in each first partition to manage data blocks; and the fine-grained cache is divided into a plurality of second partitions, and a hash table is adopted in each second partition to manage the key value pair.

5. The hotspot-aware log-structured merge-tree read-write performance optimization method of any one of claims 1 to 3, wherein the merge operation of the log-structured merge-tree comprises:

determining the level of the log structure merging tree which needs to initiate merging, calculating the heat of each data table in the level, and selecting the data table with the minimum heat to initiate merging operation;

wherein, the heat of the data table is the sum of the heat of all the data blocks in the data table; the heat of the data block is the sum of the heats of all key-value pairs in the data block.

6. The hotspot-aware log-structured merge-tree read-write performance optimization method of claim 5, wherein in the log-structured merge-tree, the heat calculation manner of the key-value pairs comprises:

judging whether the key value pair is in the fine-grained cache, if so, setting the read heat degree of the key value pair as s _r Otherwise, setting the reading heat degree to 0; s _r >0；

7. The hotspot-aware log-structured merge-tree read-write performance optimization method of claim 5, wherein the merge operation of the log-structured merge-tree further comprises:

and calculating the heat degree and the average heat degree of each data block for the new data blocks generated in the merging process, selecting the data blocks with the heat degree higher than the average heat degree, and replacing the data blocks which fail due to merging in the coarse-grained cache.

8. The hotspot-aware log-structured merge-tree read-write performance optimization method of claim 5, wherein the merge operation of the log-structured merge-tree further comprises:

and for the key value pair needing to be merged in the log structure merging tree, if the data of the updated version is cached in the fine-grained cache, the write buffer or the read-only write buffer, discarding the key value pair.

9. A storage controller, comprising: a computer-readable storage medium and a processor; the computer-readable storage medium has stored therein a computer program; the processor is configured to read a computer program in the computer-readable storage medium, and execute the hotspot-aware log-structured merge-tree read-write performance optimization method according to any one of claims 1 to 8.

10. A log-structured merge-tree based storage system, comprising: a memory, a storage device, and the storage controller of claim 9.