CN116149552A - Storage system optimization method, system, equipment and storage medium - Google Patents
Storage system optimization method, system, equipment and storage medium Download PDFInfo
- Publication number
- CN116149552A CN116149552A CN202310074098.4A CN202310074098A CN116149552A CN 116149552 A CN116149552 A CN 116149552A CN 202310074098 A CN202310074098 A CN 202310074098A CN 116149552 A CN116149552 A CN 116149552A
- Authority
- CN
- China
- Prior art keywords
- storage
- hard disk
- space
- storage hard
- bitmap
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003860 storage Methods 0.000 title claims abstract description 355
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000005457 optimization Methods 0.000 title claims abstract description 26
- 238000012544 monitoring process Methods 0.000 claims abstract description 13
- 238000001514 detection method Methods 0.000 claims description 4
- 239000002699 waste material Substances 0.000 abstract description 2
- 238000007726 management method Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000005055 memory storage Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000008187 granular material Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a storage system optimization method, a system, equipment and a storage medium, which are used for solving the technical problems that space waste and incapability of continuously providing services due to a storage system optimization strategy in the existing mixed flash scene are easy to cause. Detecting the space use condition of the first storage hard disk; triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value; distributing a required amount of storage space from a second storage hard disk to the first storage hard disk; and monitoring the use condition of the storage space in real time, and dynamically adjusting and distributing the storage space between the first storage hard disk and the second storage hard disk. The method provided by the embodiment of the application provides a new bitmap management mode among different devices, realizes a dynamic adjustment scheme of the space among different devices from the bottom layer, and can also efficiently inquire the use condition of the device space. The performance is higher, the reliability is better, and the cost is lower.
Description
Technical Field
The present disclosure relates to the field of distributed storage technologies, and in particular, to a storage system optimization method, system, device, and storage medium.
Background
The full flash environment uses a nonvolatile Memory storage pool (NVME) SSD as a data storage device, and has the advantage of high performance, but at the same time, the cost is relatively high. The common environment uses HDD (Hard Disk Drive) disks at lower cost but with more performance. The mixed flash environment uses NVME SSD to store data with higher performance requirements, such as a log, or uses a cache pool to temporarily store the data and then swipe the data down to an HDD disk, and the like, so that the mixed flash environment has the advantages of low cost and high performance.
But it is usually an OSD (Object Storage Device, a process of returning specific data in response to a client request) that manages a bare device in the full flash and normal scenarios, whereas the mixed flash scenario has a scenario in which an OSD manages a HDD bare device and an NVME SSD bare device or a part thereof. The common method is to store data for HDD, and the NVME SSD stores metadata with more access and faster speed requirements, log or cache data to a fast device under certain conditions, etc. Because the final use scenario of the storage device tends to be complex and unpredictable, the size of the log, metadata space and buffer space used in practice cannot be predicted in advance, which results in that we cannot give a scheme for OSD management of the size of the NVME SSD space in advance when deploying the cluster. The size of the NVME SSD space actually allocated to the OSD may be more or less than the traffic demand. When the NVME SSD space is first full, because the log, metadata, cache, etc. cannot be written continuously, even if the HDD has a large amount of remaining space, the HDD cannot be used continuously; similarly, when the HDD space is first written to be empty, the space cannot be used even if there is a remaining space on the NVME SSD. This leads to a problem of wasted space and failure of the cluster to continue to provide service.
Disclosure of Invention
The embodiment of the application provides a storage system optimization method, a system, equipment and a storage medium, which are used for solving the technical problems that space waste and incapability of continuously providing services due to a storage system optimization strategy in the existing mixed flash scene are easy to cause.
In one aspect, an embodiment of the present application provides a storage system optimization method, where the method includes:
detecting the space use condition of the first storage hard disk;
triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
based on the dynamic space adjustment instruction, a required amount of storage space is distributed from a second storage hard disk to the first storage hard disk;
and monitoring the use condition of the storage space of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and distributing the storage space between the first storage hard disk and the second storage hard disk.
In one implementation of the present application, the method further includes:
generating a bitmap of the first storage hard disk and the second storage hard disk;
and updating bitmaps of the first storage hard disk and the second storage hard disk in real time based on the storage space adjustment condition between the first storage hard disk and the second storage hard disk.
In one implementation manner of the present application, the updating, in real time, the bitmaps of the first storage hard disk and the second storage hard disk specifically includes:
when the second storage hard disk allocates the storage space with the required quantity to the first storage hard disk, setting the storage space allocated correspondingly on the bitmap of the second storage hard disk as used, and setting the bitmap of the storage space obtained by the first storage hard disk as available.
In one implementation of the present application, the method further includes:
when the first storage hard disk allocates a required amount of storage space to the second storage hard disk, setting the storage space allocated correspondingly on the bitmap of the first storage hard disk as used, and setting the bitmap of the storage space obtained by the second storage hard disk as available.
In one implementation of the present application, the minimum allocation unit of the bitmaps of the first storage hard disk and the second storage hard disk is 4KB.
In one implementation of the present application, the method further includes:
when dynamically adjusting the allocation storage space, sequentially inquiring from the first layer bitmap, the second layer bitmap and the third layer bitmap;
and if the first horizon map shows that the available space exists, entering the second horizon map layer corresponding to the first horizon map to traverse the free space.
In an implementation manner of the present application, the step of entering the idle space in the second layer map layer corresponding to the first layer map specifically includes:
determining that the second horizon icon displays available space;
entering the third bitmap layer corresponding to the second horizon map to traverse the idle space;
and finding the idle bit of the corresponding mark in the third bitmap layer, and further finding the corresponding idle disk space.
In another aspect, an embodiment of the present application further provides a storage system optimization system, where the system includes:
the hard disk space detection module is used for detecting the space use condition of the first storage hard disk;
the allocation instruction triggering module is used for triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
the storage space allocation module is used for allocating the required amount of storage space from the second storage hard disk to the first storage hard disk based on the dynamic space adjustment instruction;
and the space dynamic allocation module is used for monitoring the use condition of the storage spaces of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and allocating the storage spaces between the first storage hard disk and the second storage hard disk.
In addition, the embodiment of the application also provides storage system optimization equipment, which comprises:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
detecting the space use condition of the first storage hard disk;
triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
based on the dynamic space adjustment instruction, a required amount of storage space is distributed from a second storage hard disk to the first storage hard disk;
and monitoring the use condition of the storage space of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and distributing the storage space between the first storage hard disk and the second storage hard disk.
Finally, embodiments of the present application also provide a storage system optimized non-volatile computer storage medium storing computer executable instructions configured to:
detecting the space use condition of the first storage hard disk;
triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
based on the dynamic space adjustment instruction, a required amount of storage space is distributed from a second storage hard disk to the first storage hard disk;
and monitoring the use condition of the storage space of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and distributing the storage space between the first storage hard disk and the second storage hard disk.
The storage system optimization method, the system, the equipment and the storage medium provided by the embodiment of the application have the following beneficial effects:
(1) The performance is better. The scheme supports OSD space dynamic management in a mixed flash scene, enables data with relatively high time delay requirements, such as logs, metadata, caches and the like, to be placed on NVME SSD, and greatly improves the performance of storage.
(2) And the reliability is higher. The scheme avoids the problem that as the water level of the stored data increases, a certain device in the OSD is fully written first, so that the whole OSD cannot provide service.
(3) And the cost is lower. The scheme solves the problem brought by equipment management in the mixed flash scene, fully utilizes all equipment spaces, and reduces the additional cost consumption such as storage capacity expansion and the like. And the budget increase caused by the fact that the storage performance of the full HDD does not meet the requirement of full flash of the replacement can also be avoided.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
FIG. 1 is a flowchart of a method for optimizing a storage system according to an embodiment of the present application;
fig. 2 is a schematic diagram of bitmap management provided in an embodiment of the present application;
FIG. 3 is a diagram of a storage system optimization system according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram of a storage system optimizing device according to an embodiment of the present application.
Detailed Description
For the purposes, technical solutions and advantages of the present application, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The full flash environment uses a nonvolatile Memory storage pool (NVME) SSD as a data storage device, and has the advantage of high performance, but at the same time, the cost is relatively high. The common environment uses HDD (Hard Disk Drive) disks at lower cost but with more performance. The mixed flash environment uses NVME SSD to store data with higher performance requirements, such as a log, or uses a cache pool to temporarily store the data and then swipe the data down to an HDD disk, and the like, so that the mixed flash environment has the advantages of low cost and high performance.
But it is usually an OSD (Object Storage Device, a process of returning specific data in response to a client request) that manages a bare device in the full flash and normal scenarios, whereas the mixed flash scenario has a scenario in which an OSD manages a HDD bare device and an NVME SSD bare device or a part thereof. The common method is to store data for HDD, and the NVME SSD stores metadata with more access and faster speed requirements, log or cache data to a fast device under certain conditions, etc. Because the final use scenario of the storage device tends to be complex and unpredictable, the size of the log, metadata space and buffer space used in practice cannot be predicted in advance, which results in that we cannot give a scheme for OSD management of the size of the NVME SSD space in advance when deploying the cluster. The size of the NVME SSD space actually allocated to the OSD may be more or less than the traffic demand. When the NVME SSD space is first full, because the log, metadata, cache, etc. cannot be written continuously, even if the HDD has a large amount of remaining space, the HDD cannot be used continuously; similarly, when the HDD space is first written to be empty, the space cannot be used even if there is a remaining space on the NVME SSD. This leads to a problem of wasted space and failure of the cluster to continue to provide service.
The embodiment of the application provides a space management scheme capable of dynamically adjusting an HDD space and an NVME SSD space for OSD management based on a storage system in a mixed flash scene. The configuration items provided to the developer determine the conditions under which the space triggers are dynamically adjusted and the granularity of the space is dynamically adjusted each time.
The embodiment of the application provides a scheme for managing the space of the HDD and the NVME SSD by the OSD, and effectively solves the problem of space utilization. The following describes in detail the technical solution proposed in the embodiments of the present application through the accompanying drawings.
The embodiment of the application specifically provides a storage system optimization method, a system, equipment and a storage medium. Fig. 1 is a flowchart of a storage system optimization method according to an embodiment of the present application. As shown in fig. 1, the method mainly comprises the following steps:
step 101, detecting the space use condition of the first storage hard disk.
In the embodiment of the application, the design concept is that when the storage device writes more metadata or data such as logs and the like to cause insufficient space of the first storage hard disk, and the second storage hard disk still has a large amount of space, the condition for triggering dynamic space adjustment is met, and a certain amount of space is allocated from the second storage hard disk to be used as the space of the first storage hard disk; similarly, when the second storage hard disk writes more data, and the condition that the space is insufficient to trigger the dynamic space adjustment is met, a certain amount of space is allocated from the first storage hard disk as the space of the second storage hard disk for use.
It should be noted that, in the embodiment of the present application, the first storage hard disk corresponds to a hard disk in an NVME SSD space, and the second storage hard disk corresponds to a hard disk in an HDD space.
In the embodiment of the application, the storage space is allocated from the storage hard disk with the free storage space to the storage hard disk with the insufficient storage space based on the corresponding trigger instruction. Therefore, the system can set a corresponding detection mechanism to detect the residual space of all the storage hard disks in the system, and when the residual space of the first storage hard disk is smaller than a first preset threshold value, a corresponding dynamic space adjustment instruction is generated and triggered.
And step 103, distributing a required amount of storage space from a second storage hard disk to the first storage hard disk based on the dynamic space adjustment instruction.
In this embodiment of the present application, corresponding to the dynamic space adjustment instruction, the system automatically allocates a storage space in the second storage hard disk to the first storage hard disk. Before that, the system calculates the size of the storage space actually required by the first storage hard disk at this time, then divides the calculated storage space actually required in the second storage hard disk, and then performs the storage space allocation operation.
And 104, monitoring the use condition of the storage spaces of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and distributing the storage spaces between the first storage hard disk and the second storage hard disk.
In this embodiment, after space allocation is completed, the usage conditions of the storage space of the first storage hard disk and the second storage hard disk may be monitored in real time. When the newly allocated storage space of the first storage hard disk is used up again, a certain amount of storage space is allocated again from the remaining storage space of the second storage hard disk. When the remaining storage space of the second storage hard disk is insufficient to provide to the first storage hard disk, the allocation can be performed from the other storage hard disks.
In this embodiment of the present application, when the first storage hard disk is used as a storage hard disk with a higher use frequency, the allocated resource may generate a bitmap generated by mapping the corresponding storage hard disk.
Further, based on the storage space adjustment condition between the first storage hard disk and the second storage hard disk, the bitmaps of the first storage hard disk and the second storage hard disk are updated in real time.
Further, the bitmaps of the first storage hard disk and the second storage hard disk are updated in real time, specifically: when the second storage hard disk allocates the storage space with the required quantity to the first storage hard disk, setting the storage space allocated correspondingly on the bitmap of the second storage hard disk as used, and setting the bitmap of the storage space obtained by the first storage hard disk as available.
When the first storage hard disk allocates a required amount of storage space to the second storage hard disk, setting the storage space allocated correspondingly on the bitmap of the first storage hard disk as used, and setting the bitmap of the storage space obtained by the second storage hard disk as available.
In this embodiment of the present application, the minimum allocation unit of the bitmap of the first storage hard disk and the bitmap of the second storage hard disk is 4KB. The reason is that, on one hand, the minimum allocation unit of the solid-state flash granule device is 4K, on the other hand, the addressing delay consumption of the HDD disk is larger, and in order to reduce frequent disk addressing in a general storage scheme, larger space units such as integer multiples of 4K are often used as allocation units of the HDD device.
In the embodiment of the application, when the allocation storage space is dynamically adjusted, sequentially inquiring from the first layer bitmap, the second layer bitmap and the third layer bitmap; and if the first horizon map shows that the available space exists, entering the second horizon map layer corresponding to the first horizon map to traverse the free space.
In this embodiment, entering the traversal empty space in the second layer map layer corresponding to the first layer map specifically includes: determining that the second horizon icon displays available space; entering the third bitmap layer corresponding to the second horizon map to traverse the idle space; and finding the idle bit of the corresponding mark in the third bitmap layer, and further finding the corresponding idle disk space.
In order to find the use situation of the space more efficiently, the embodiment of the application uses a three-layer bitmap to manage the device space. Using array preservation, each bit of the array index represents the use of a block of space or a series of spaces. And when the space is allocated, the space is sequentially inquired from a high layer to a bottom layer, so that the performance consumption caused by traversing the bitmap is reduced. When the system is started or fault reverts, the bitmap information is read from the log to the memory and then the memory is resident, so that the system access efficiency is greatly improved. Bitmap management schematic, as shown in fig. 2. The manner in which the device space is managed using the tri-layer bitmap can reduce the performance consumption of traversing the bitmap.
In the embodiment of the present application, in order to reduce the resource occupation of the system, in an actual implementation, dynamic space balancing may be triggered only when a certain device cannot allocate space. It is not necessary to trigger dynamic spatial equalization using background scanning that the current device space is below a certain threshold. The size of the space allocated for each dynamic spatial equalization is determined by the developer configuration item.
The embodiment of the application provides a space management scheme capable of dynamically adjusting an HDD space and an NVME SSD space for OSD management based on a storage system in a mixed flash scene. The problems of service interruption and the like caused by insufficient space of a certain single device are avoided. The embodiment of the application provides a new bitmap management mode among different devices, realizes a dynamic adjustment scheme of the space among different devices from the bottom layer, and can also efficiently inquire the use condition of the device space.
According to the storage system optimization method, OSD space dynamic management in a mixed flash scene is supported, so that data with more access time delay requirements such as logs, metadata and caches are placed on an NVME SSD, and the performance of storage is greatly improved. The problem that the whole OSD cannot provide service due to the fact that a certain device in the OSD is fully written along with the increase of the water level of stored data is avoided, and the reliability is higher. The scheme solves the problem brought by equipment management in the mixed flash scene, fully utilizes all equipment spaces, and reduces the additional cost consumption such as storage capacity expansion and the like. And the budget increase caused by the fact that the storage performance of the full HDD does not meet the requirement of full flash of the replacement can also be avoided.
The foregoing is a storage system optimization method provided by the embodiment of the present application, and based on the same inventive concept, the embodiment of the present application further provides a storage system optimization system, and fig. 3 is a composition diagram of the storage system optimization system provided by the embodiment of the present application, as shown in fig. 3, where the system mainly includes:
a hard disk space detection module 301, configured to detect a space usage condition of a first storage hard disk;
the allocation instruction triggering module 302 is configured to trigger a dynamic adjustment space instruction when the remaining space of the first storage hard disk is less than a first preset threshold;
a storage space allocation module 303, configured to allocate a required amount of storage space from a second storage hard disk to the first storage hard disk based on the dynamic adjustment space command;
the space dynamic allocation module 304 is configured to monitor the storage space usage of the first storage hard disk and the second storage hard disk in real time, and dynamically adjust and allocate the storage space between the first storage hard disk and the second storage hard disk.
In this embodiment of the present application, the storage space allocation module 303 specifically performs: firstly, calculating the size of the storage space actually required by the first storage hard disk, then dividing the calculated storage space actually required in the second storage hard disk, and then executing the allocation operation of the storage space.
In this embodiment, the spatial dynamic allocation module 304 specifically performs: and monitoring the use condition of the storage space of the first storage hard disk and the second storage hard disk in real time. When the newly allocated storage space of the first storage hard disk is used up again, a certain amount of storage space is allocated again from the remaining storage space of the second storage hard disk.
The space dynamic allocation module 304 further includes a bitmap display sub-module, and the bitmap display sub-module executes: and updating bitmaps of the first storage hard disk and the second storage hard disk in real time based on the storage space adjustment condition between the first storage hard disk and the second storage hard disk.
Specifically, the bitmap display submodule specifically performs: when the second storage hard disk allocates the storage space with the required quantity to the first storage hard disk, setting the storage space allocated correspondingly on the bitmap of the second storage hard disk as used, and setting the bitmap of the storage space obtained by the first storage hard disk as available. When the first storage hard disk allocates a required amount of storage space to the second storage hard disk, setting the storage space allocated correspondingly on the bitmap of the first storage hard disk as used, and setting the bitmap of the storage space obtained by the second storage hard disk as available.
Further, when the bitmap display sub-module dynamically adjusts and allocates the storage space, sequentially inquiring from the first layer bitmap, the second layer bitmap and the third layer bitmap; and if the first horizon map shows that the available space exists, entering the second horizon map layer corresponding to the first horizon map to traverse the free space. Determining that the second horizon icon displays available space; then entering the third bitmap layer corresponding to the second bitmap layer to traverse the idle space; and finding the idle bit of the corresponding mark in the third bitmap layer, and further finding the corresponding idle disk space.
According to the storage system optimization system, OSD space dynamic management in a mixed flash scene is supported, so that data with more access time delay requirements such as logs, metadata and caches are placed on an NVME SSD, and the performance of storage is greatly improved. The problem that the whole OSD cannot provide service due to the fact that a certain device in the OSD is fully written along with the increase of the water level of stored data is avoided, and the reliability is higher. The scheme solves the problem brought by equipment management in the mixed flash scene, fully utilizes all equipment spaces, and reduces the additional cost consumption such as storage capacity expansion and the like. And the budget increase caused by the fact that the storage performance of the full HDD does not meet the requirement of full flash of the replacement can also be avoided.
The foregoing is a storage system optimizing system provided by the embodiment of the present application, and based on the same inventive concept, the embodiment of the present application further provides a storage system optimizing device, and fig. 4 is a schematic diagram of a storage system optimizing device provided by the embodiment of the present application, as shown in fig. 4, where the device mainly includes: at least one processor 401; and a memory 402 communicatively coupled to the at least one processor; wherein the memory 402 stores instructions executable by the at least one processor 401, the instructions being executable by the at least one processor 401 to enable the at least one processor 401 to: detecting the space use condition of the first storage hard disk;
triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
based on the dynamic space adjustment instruction, a required amount of storage space is distributed from a second storage hard disk to the first storage hard disk;
and monitoring the use condition of the storage space of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and distributing the storage space between the first storage hard disk and the second storage hard disk.
In addition, the embodiments of the present application further provide a nonvolatile computer storage medium storing computer executable instructions configured to: detecting the space use condition of the first storage hard disk;
triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
based on the dynamic space adjustment instruction, a required amount of storage space is distributed from a second storage hard disk to the first storage hard disk;
and monitoring the use condition of the storage space of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and distributing the storage space between the first storage hard disk and the second storage hard disk.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
All embodiments in the application are described in a progressive manner, and identical and similar parts of all embodiments are mutually referred, so that each embodiment mainly describes differences from other embodiments. In particular, for the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments in part.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.
Claims (10)
1. A method of optimizing a storage system, the method comprising:
detecting the space use condition of the first storage hard disk;
triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
based on the dynamic space adjustment instruction, a required amount of storage space is distributed from a second storage hard disk to the first storage hard disk;
and monitoring the use condition of the storage space of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and distributing the storage space between the first storage hard disk and the second storage hard disk.
2. The storage system optimization method of claim 1, further comprising:
generating a bitmap of the first storage hard disk and the second storage hard disk;
and updating bitmaps of the first storage hard disk and the second storage hard disk in real time based on the storage space adjustment condition between the first storage hard disk and the second storage hard disk.
3. The storage system optimization method according to claim 2, wherein the updating the bitmaps of the first storage hard disk and the second storage hard disk in real time is specifically:
when the second storage hard disk allocates the storage space with the required quantity to the first storage hard disk, setting the storage space allocated correspondingly on the bitmap of the second storage hard disk as used, and setting the bitmap of the storage space obtained by the first storage hard disk as available.
4. A method of optimizing a storage system according to claim 3, said method further comprising:
when the first storage hard disk allocates a required amount of storage space to the second storage hard disk, setting the storage space allocated correspondingly on the bitmap of the first storage hard disk as used, and setting the bitmap of the storage space obtained by the second storage hard disk as available.
5. The method of claim 2, wherein the bitmap minimum allocation unit of the first storage hard disk and the second storage hard disk is 4KB.
6. The storage system optimization method of claim 1, further comprising:
when dynamically adjusting the allocation storage space, sequentially inquiring from the first layer bitmap, the second layer bitmap and the third layer bitmap;
and if the first horizon map shows that the available space exists, entering the second horizon map layer corresponding to the first horizon map to traverse the free space.
7. The storage system optimization method according to claim 6, wherein the step of entering the second layer bitmap layer corresponding to the first layer bitmap traverses a free space, specifically:
determining that the second horizon icon displays available space;
entering the third bitmap layer corresponding to the second horizon map to traverse the idle space;
and finding the idle bit of the corresponding mark in the third bitmap layer, and further finding the corresponding idle disk space.
8. A storage system optimization system, the system comprising:
the hard disk space detection module is used for detecting the space use condition of the first storage hard disk;
the allocation instruction triggering module is used for triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
the storage space allocation module is used for allocating the required amount of storage space from the second storage hard disk to the first storage hard disk based on the dynamic space adjustment instruction;
and the space dynamic allocation module is used for monitoring the use condition of the storage spaces of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and allocating the storage spaces between the first storage hard disk and the second storage hard disk.
9. A storage system optimization device, the device comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
detecting the space use condition of the first storage hard disk;
triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
based on the dynamic space adjustment instruction, a required amount of storage space is distributed from a second storage hard disk to the first storage hard disk;
and monitoring the use condition of the storage space of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and distributing the storage space between the first storage hard disk and the second storage hard disk.
10. A non-volatile computer storage medium storing computer executable instructions for storage system optimization, the computer executable instructions configured to:
detecting the space use condition of the first storage hard disk;
triggering a dynamic space adjustment instruction when the residual space of the first storage hard disk is smaller than a first preset threshold value;
based on the dynamic space adjustment instruction, a required amount of storage space is distributed from a second storage hard disk to the first storage hard disk;
and monitoring the use condition of the storage space of the first storage hard disk and the second storage hard disk in real time, and dynamically adjusting and distributing the storage space between the first storage hard disk and the second storage hard disk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310074098.4A CN116149552A (en) | 2023-01-30 | 2023-01-30 | Storage system optimization method, system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310074098.4A CN116149552A (en) | 2023-01-30 | 2023-01-30 | Storage system optimization method, system, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116149552A true CN116149552A (en) | 2023-05-23 |
Family
ID=86357797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310074098.4A Pending CN116149552A (en) | 2023-01-30 | 2023-01-30 | Storage system optimization method, system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116149552A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116578247A (en) * | 2023-07-10 | 2023-08-11 | 西安羚控电子科技有限公司 | Metadata-based data access method and system |
-
2023
- 2023-01-30 CN CN202310074098.4A patent/CN116149552A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116578247A (en) * | 2023-07-10 | 2023-08-11 | 西安羚控电子科技有限公司 | Metadata-based data access method and system |
| CN116578247B (en) * | 2023-07-10 | 2023-10-13 | 西安羚控电子科技有限公司 | Metadata-based data access method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8069191B2 (en) | Method, an apparatus and a system for managing a snapshot storage pool | |
| US9361034B2 (en) | Transferring storage resources between snapshot storage pools and volume storage pools in a distributed network | |
| US7971025B2 (en) | Method and apparatus for chunk allocation in a thin provisioning storage system | |
| US8495318B2 (en) | Memory page management in a tiered memory system | |
| US10387038B1 (en) | Storage space allocation for logical disk creation | |
| US9459806B2 (en) | Combining virtual mapping metadata and physical space mapping metadata | |
| US10152340B2 (en) | Configuring cache for I/O operations of virtual machines | |
| US20140304452A1 (en) | Method for increasing storage media performance | |
| US9639459B2 (en) | I/O latency and IOPs performance in thin provisioned volumes | |
| US9823875B2 (en) | Transparent hybrid data storage | |
| US10534720B2 (en) | Application aware memory resource management | |
| KR20110117099A (en) | Mapping address table maintenance in a memory device | |
| US9983806B2 (en) | Storage controlling apparatus, information processing apparatus, and computer-readable recording medium having stored therein storage controlling program | |
| WO2015114643A1 (en) | Data storage system rebuild | |
| KR20190063378A (en) | Dynamic cache partition manager in heterogeneous virtualization cloud cache environment | |
| CN116149552A (en) | Storage system optimization method, system, equipment and storage medium | |
| US11144445B1 (en) | Use of compression domains that are more granular than storage allocation units | |
| US9710183B2 (en) | Effectively limitless apparent free space on storage device | |
| CN114518962A (en) | Memory management method and device | |
| US10140038B2 (en) | Utilization of a thin provision grid storage array | |
| US11176047B2 (en) | Data storage system with physical storage and cache memory | |
| US10992751B1 (en) | Selective storage of a dataset on a data storage device that is directly attached to a network switch | |
| CN114706628B (en) | Data processing method and device of distributed storage system based on one pool and multiple cores | |
| CN118363875A (en) | Memory recovery method, device, equipment, medium and product | |
| KR20240127860A (en) | Systems and methods for data storage management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |