CN111124303A - Data storage method, device and system - Google Patents
Data storage method, device and system Download PDFInfo
- Publication number
- CN111124303A CN111124303A CN201911310870.8A CN201911310870A CN111124303A CN 111124303 A CN111124303 A CN 111124303A CN 201911310870 A CN201911310870 A CN 201911310870A CN 111124303 A CN111124303 A CN 111124303A
- Authority
- CN
- China
- Prior art keywords
- data
- stored
- access frequency
- hard disk
- management area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000013500 data storage Methods 0.000 title claims abstract description 50
- 238000013523 data management Methods 0.000 claims abstract description 94
- 230000003287 optical effect Effects 0.000 claims abstract description 68
- 230000015654 memory Effects 0.000 claims description 22
- 238000004140 cleaning Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 2
- 230000003139 buffering effect Effects 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000002035 prolonged effect Effects 0.000 abstract description 2
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
The invention discloses a data storage method, a device and a system, wherein the method comprises the following steps: acquiring data to be stored, and storing attribute information of the data to be stored in a data management area; judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency, modifying the attribute information of the data in the data management area when the access frequency is greater than or equal to the preset access frequency, and storing the hot data into the hard disk; acquiring the storage position of the thermal data in the hard disk, and storing the storage position information into a data management area; writing all data to be stored into an optical disk library; and acquiring the storage positions of all the data to be stored in the optical disk library, and storing the storage position information into the data management area. By implementing the invention, different steps are adopted to store data in combination with different attribute information of the data to be stored, the problem that the data must be firstly subjected to magnetic buffering and/or electric buffering and then stored in an optical disk in the prior data storage technology is solved, and the service life of media such as a magnetic disk and the like is prolonged.
Description
Technical Field
The invention relates to the field of data storage, in particular to a data storage method, device and system.
Background
In the big data era, a large number of enterprises store a large amount of data, and the data can be classified and stored in different media according to the application frequency and application scenes of different data, so that the cost is saved. In the existing storage technology, the mode of combining a disk array with a tape library for data storage is still the mainstream technology which can be applied to data center data storage. After a series of processing, all the original data are finally stored in the optical disc for storage. Because the media such as the magnetic disk and the like are limited by the number of times of reading and writing, when a large amount of data is stored, the service life of the magnetic disk is seriously influenced by reading and writing for many times, that is, the existing storage technology must firstly subject the data to magnetic buffering and/or electric buffering and then store the data into the optical disk, and the service life of the media such as the magnetic disk and the like is influenced by the storage mode.
Disclosure of Invention
Therefore, the technical problem to be solved by the present invention is to overcome the defect that the service life of media such as a magnetic disk and the like is shortened by first magnetically buffering and/or electrically buffering data and then storing the data in the optical disk in the existing data storage technology, thereby providing a data storage method, device and system.
According to a first aspect, an embodiment of the present invention discloses a data storage method, including: acquiring data to be stored, wherein the attribute of the data to be stored is defaulted to cold data, and the attribute of the data comprises hot data or cold data; storing the attribute information of the data to be stored into a data management area, wherein the data management area is a storage space arranged in a memory; judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency or not, wherein the data with the access frequency greater than or equal to the preset access frequency are hot data, and the data with the access frequency less than the preset access frequency are cold data; when the data to be stored is thermal data, storing the thermal data into a hard disk, and modifying the attribute information of the data to be stored in the data management area; acquiring the storage position of the thermal data in a hard disk, and storing the storage position information of the thermal data in the hard disk into the data management area; writing all the data to be stored into an optical disk library; and acquiring the storage positions of all the data to be stored in the optical disc library, and storing the storage position information of all the data to be stored in the optical disc library into the data management area.
With reference to the first aspect, in a first implementation manner of the first aspect, after the data to be stored is obtained, the method for storing the data further includes: and marking the data to be stored, and storing the identification information of the data to be stored into the data management area, wherein the identification information of the data to be stored is unique.
With reference to the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the method for reading data further includes: judging whether the current access frequency of the hot data stored in the hard disk is lower than a preset access frequency or not according to a preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, cleaning the hot data in the hard disk, feeding back cleaning information of the hot data to a data management area, modifying attribute information of the hot data, and deleting the hard disk storage position information of the hot data in the data management area.
According to a second aspect, an embodiment of the present invention discloses a data reading method, including: acquiring data to be read, and judging whether the attribute of the data to be read is hot data or not; the thermal data refers to data stored in the hard disk by the data storage method as described in the first aspect or any embodiment of the first aspect; when the data to be read is hot data, directly acquiring the hard disk storage position information of the hot data in the data management area; and reading the thermal data according to the storage position information of the hard disk.
With reference to the second aspect, in a first implementation manner of the second aspect, the method for reading data further includes: when the data is cold data, acquiring storage position information of an optical disk library of the cold data in a data management area; acquiring the cold data according to the storage position information of the optical disk library; and reading the obtained cold data into a hard disk, and reading and processing.
With reference to the first embodiment of the second aspect, in the second embodiment of the second aspect, the method for reading data further includes: judging whether the current access frequency of the hot data stored in the hard disk is lower than a preset access frequency or not according to a preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, cleaning the hot data in the hard disk, feeding back cleaning information of the hot data to a data management area, modifying attribute information of the hot data, and deleting the hard disk storage position information of the hot data in the data management area.
According to a third aspect, an embodiment of the present invention discloses a data storage device, including: the device comprises a first acquisition module, a second acquisition module and a data management module, wherein the first acquisition module is used for acquiring data to be stored, the attribute of the data to be stored is default to be cold data, the attribute of the data comprises hot data or cold data, and the first acquisition module is also used for storing the attribute information of the data to be stored into a data management area, and the data management area is a storage space arranged in a memory; the first judging module is used for judging whether the access frequency of the data to be stored is greater than or equal to the data of the preset access frequency, the data of which the access frequency is greater than or equal to the preset access frequency are hot data, and the data of which the access frequency is less than the preset access frequency are cold data; the first storage module is used for storing the hot data into a hard disk and modifying the attribute information of the data to be stored in the data management area when the data to be stored is the hot data; the second acquisition module is used for acquiring the storage position of the thermal data in the hard disk and storing the storage position information of the thermal data in the hard disk into the data management area; the second storage module is used for writing all the data to be stored into an optical disk library; and the third acquisition module is used for acquiring the storage positions of all the data to be stored in the optical disc library and storing the storage position information of all the data to be stored in the optical disc library into the data management area.
According to a fourth aspect, an embodiment of the present invention discloses a data reading apparatus, including: the fourth acquisition module is used for acquiring data to be read; the second judgment module is used for judging whether the attribute of the data to be read is hot data or not; the thermal data refers to data stored in the hard disk by the data storage device according to the third aspect; a fifth obtaining module, configured to directly obtain, when the data to be read is hot data, location information of a hard disk storage of the hot data in the data management area; and the reading module is used for reading the thermal data according to the storage position information of the hard disk.
According to a fifth aspect, an embodiment of the present invention discloses a data storage system, including: at least one control device, configured to execute the data storage method according to the first aspect or any implementation manner of the first aspect, and store the received data to be stored.
According to a sixth aspect, an embodiment of the present invention discloses a data reading system, including: at least one control device for executing the data reading method according to the second aspect or any embodiment of the second aspect to read data.
According to a seventh aspect, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the method for storing data according to the first aspect or any embodiment of the first aspect or the method for reading data according to the second aspect or any embodiment of the second aspect.
The technical scheme of the invention has the following advantages:
1. according to the data storage method provided by the embodiment of the invention, the access frequency of the data to be stored is judged, the attribute information of the data to be stored is correspondingly modified, when the access frequency is greater than the preset access frequency, the data is thermal data, the thermal data is stored in the hard disk, the position information of the thermal data in the hard disk is stored in the data management area, and finally all the data is stored in the optical disk library to finish the data storage work.
2. The data reading method provided by the embodiment of the invention comprises the steps of firstly obtaining data to be read, judging whether the attribute of the data to be read is thermal data or not, and directly reading the thermal data by directly obtaining the storage position information of a hard disk of the thermal data in a data management area when the data to be read is the thermal data; when the data is judged to be cold data, the cold data can be directly obtained through obtaining the information of the storage position of the optical disk library of the cold data in the data management area, reading and processing are carried out, and the data can be directly obtained through different position information of the data by combining the reading mode of the data and the access frequency of the data to be read, so that the data reading speed is improved, and the consumption of storage media such as a magnetic disk is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic diagram of the structure of the data management area, the optical disc library, and the data management area according to the present invention;
fig. 2 is a flowchart of a specific example of a data storage method according to embodiment 1 of the present invention;
fig. 3 is a flowchart of a specific example of a data reading method according to embodiment 2 of the present invention;
fig. 4 is a block diagram showing a specific example of a data storage device according to embodiment 3 of the present invention;
fig. 5 is a block diagram showing a specific example of a data reading apparatus according to embodiment 4 of the present invention;
FIG. 6 is a schematic block diagram of a control device of the present invention;
fig. 7 is a functional block diagram of a first controller in the control apparatus of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "first", "second", and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; the two elements may be directly connected or indirectly connected through an intermediate medium, or may be communicated with each other inside the two elements, or may be wirelessly connected or wired connected. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
As shown in fig. 1, the data storage method provided in the embodiment of the present invention may be applied to an application environment composed of a data management area 10, an optical disc library 20, and a hard disc storage area 30, where the data management area 10 is used to store attribute information and location information of data, and manage data stored in the optical disc library and the hard disc; the optical disc library 20 is used for storing data, realizing the read-write operation of the optical disc library through an optical disc drive, realizing the automatic access of optical discs through a mechanical device and automatically expanding the capacity of the optical disc library; the Hard Disk storage area is used for storing data with high access frequency and also for data cleaning, and specifically may be a Hard Disk array using a Solid State Drive (SSD) or a mechanical Hard Disk (Hard Disk Drive, HDD) as a medium.
Example 1
The embodiment of the present invention provides a data storage method, which can be applied to a specific application scenario where large-scale organizations such as banks, governments, and companies need to store data with different access frequencies, and the data storage method in this embodiment, as shown in fig. 2, includes:
step S11: acquiring data to be stored, defaulting the attribute of the data to be stored into cold data, storing the attribute information of the data to be stored into a data management area, wherein the attribute information of the data to be stored comprises hot data or cold data, and the data management area is a storage space arranged in an internal memory; in this embodiment, the data to be stored may be data information of assets, receipt and payment details, and the like of a client in a bank system, or data information of meeting records, policy changes, and the like in a government system, or data information of purchasing, expenditure, and the like in a company system, where the data all need to be stored as soon as possible after the data is generated, specifically, the data information may be acquired through a Universal Serial Bus (USB) by a manner of acquiring the data information, a data line may be connected to the system, the acquired data content may be transmitted through a USB data line, or may be directly acquired through a Wi-Fi system, and a technician in the art may determine a manner of acquiring the data according to actual needs, which is not limited in the embodiment of the present invention, specifically, all the data to be stored is default cold data, and the cold data is data information with an access frequency lower than a preset access frequency, the attribute information of all the data to be stored may be stored in the data management area in advance.
Step S12: judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency or not, wherein the data with the access frequency greater than or equal to the preset access frequency are hot data, and the data with the access frequency less than the preset access frequency are cold data; in this embodiment, the access frequency is the number of times that the user calls the data in the target time period, and the attribute information of the data may be hot data or cold data, specifically, when the attribute information of the data is hot data, it indicates that the access frequency of the data is higher than a preset access frequency threshold, that is, in a specific application scenario, the data may be called multiple times in the target time period, for example, in a bank system, since the user checks the balance information multiple times in the target time period, the data storing the balance of the user is called multiple times, and therefore, the data storing the balance of the user is hot data; when the attribute of the data is cold data, it indicates that the access frequency of the data in the target time period is lower than the preset access frequency threshold, that is, the number of times the data is used in the target time period is lower than the preset number of times.
Step S13: and when the data to be stored is thermal data, storing the thermal data into the hard disk, and modifying the attribute information of the data to be stored in the data management area.
Exemplarily, that is, attribute information determined as hot data by judging the access frequency is modified from cold data to hot data in the data management area; the data management area 10, which may also be referred to as a storage management area, is a management area provided in the memory and is used for managing data that has already been stored; when the data to be stored is thermal data, the attribute information indicating the identity of the thermal data is stored in the data management area 10, and the data information carried by the thermal data is stored in a hard disk, specifically, the hard disk can be a nonvolatile memory used on a mobile terminal.
Step S14: and acquiring the storage position of the thermal data in the hard disk, and storing the storage position information of the thermal data in the hard disk into the data management area. Illustratively, after the thermal data is stored in the hard disk, the storage location of the thermal data in the hard disk is obtained, since the hard disk can store a large amount of data, after the thermal data is stored in the hard disk, the storage location of the thermal data in the hard disk needs to be obtained, and the storage location information of the thermal data in the hard disk is stored in the data management area 10, so as to facilitate the subsequent management operation of the data management area 10 on the stored data.
In particular, the storage locations in a hard disk may be n tracks of m sectors, e.g., a hard disk consisting of a plurality of platters that can store information on both the top and bottom surfaces, and thus the surfaces of the platters are referred to as recording surfaces having a series of concentric circles, referred to as tracks. Each recording surface typically has tens to hundreds of tracks, each of which is divided into sectors. The addresses of the magnetic tracks are numbered from outside to inside in sequence, the outermost concentric circle is called a magnetic track 0, and the innermost concentric circle is called a magnetic track n. Each track is divided into sectors. The sector number may be a serial number or an interval number. After the recording surface is numbered by the above process, the storage position in the hard disk with n tracks and m sectors, that is, the hard disk address can be used for finding the corresponding data storage position in the actual hard disk.
Step S15: and writing all the data to be stored into the optical disk library. For example, all the data to be stored includes hot data and cold data, the optical disc library 20 may be an optical disc library capable of expanding capacity, capacity expansion may be achieved by sequentially placing a plurality of optical discs in the same optical disc library, specifically, automatic access operation of a plurality of optical discs is completed by a mechanical device in the optical disc library, and high-speed optical disc storage and reading operation may also be achieved by using a large number of optical disc drives.
Step S16: and acquiring the storage positions of all the data to be stored in the optical disc library, and storing the storage position information of all the data to be stored in the optical disc library into the data management area. Illustratively, when all data is stored in the optical disc library 20, since there are a plurality of optical discs in the optical disc library, and the plurality of optical discs are placed in sequence, the storage location of the optical disc at this time of the data can be represented by the n number optical disc, and when the storage location information of each piece of data is obtained, the storage location information of this piece of data is stored in the data management area 10, which facilitates the management operation of the data management area 10 on all stored data.
The data storage method provided by the embodiment of the invention is characterized in that the attribute information of the data to be stored is judged, namely the access frequency of the data to be stored is judged, the attribute judgment result information of the data to be stored is stored in the data management area, when the access frequency is greater than the preset access frequency, the data is determined to be thermal data, the thermal data is stored in the hard disk, the position information of the thermal data in the hard disk is stored in the data management area, and finally all the data are stored in the optical disk library to finish the data storage work, thereby solving the problem that the data in the existing data storage technology must be firstly magnetically buffered and/or electrically buffered, namely the data is firstly stored in a magnetic storage medium and then is slowly recorded in an optical disk, combining the attribute information of the data, selecting different storage paths and simplifying the data storage step, the service life of the storage media such as the magnetic disk is prolonged.
As an optional implementation manner of the present application, the method for storing data provided in the embodiment of the present invention further includes: after the data to be stored is obtained, the data to be stored is identified, and the identification information of the data to be stored is stored in the data management area, wherein the identification information of the data to be stored is unique. Illustratively, data can be identified by an Identity (ID), each data can correspond to a unique identification code, and the identification code can be a data number or an alphabetical code. According to the data storage method provided by the embodiment of the invention, the unique coding information is used for identifying the data to be stored, so that the data can be conveniently and uniformly managed in the subsequent data operation. As an optional implementation manner of the present application, the method for storing data provided in the embodiment of the present invention further includes:
judging whether the current access frequency of the hot data stored in the hard disk is lower than the preset access frequency or not according to the preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, cleaning the hot data in the hard disk, feeding back cleaning information of the hot data to the data management area 10, modifying attribute information of the hot data, and deleting the hard disk storage location information of the hot data in the data management area 10.
For example, the current access frequency of the thermal data in the hard disk is judged according to a preset time interval, where the preset time interval may be 5 seconds or 10 seconds, and a person skilled in the art may determine the current access frequency according to a specific situation, which is not limited by the present invention.
According to the data storage method provided by the embodiment of the invention, the access frequency of the hot data stored in the hard disk is judged regularly, and when the access frequency becomes lower and is lower than the preset access frequency, the data is cleaned. And the attribute information of the data is modified in the data management area, the position information of the hard disk corresponding to the data in the data management area is deleted, and the data which does not accord with the hot data standard in the hard disk is deleted, so that the purpose of saving the hard disk space is achieved, and the cost of data storage is reduced.
Example 2
The embodiment of the present invention provides a data reading method, which can be applied to a specific application scenario in which a mobile terminal or a service person in a large-scale organization such as a bank, a government, a company, etc. needs to read stored data, and the data reading method in this embodiment, as shown in fig. 3, includes:
step S21: acquiring data to be read; in this embodiment, the data to be read may be the data stored in the above embodiment, for example, when the user queries the balance information, the user balance data stored in advance needs to be read, so as to obtain the current balance of the user.
Step S22: judging whether the attribute of the data to be read is hot data or not; the thermal data refers to data stored in the hard disk by the data storage method according to any of the above embodiments. For example, the attribute information of the data may be hot data or cold data, and when the data is hot data, the data indicates that the data is data stored in the hard disk because the access frequency is greater than or equal to the preset access frequency; when the data is cold data, it indicates that the data is data stored in the optical disc library 20 because the access frequency is less than the preset access frequency.
Step S23: when the data to be read is hot data, the hard disk storage position information of the hot data in the data management area is directly acquired. In this embodiment, when the data to be read is hot data, according to the data storage method described in any of the above embodiments, the hot data is stored in the hard disk, and the hard disk storage location information of the hot data is stored in the data management area 10, and at this time, to acquire the data information of the hot data, it is completed by acquiring the storage location information of the hot data in the data management area 10.
Step S24: and reading the hot data according to the storage position information of the hard disk. In this embodiment, the hard disk storage location information may be n-track m sectors, which indicates the specific location of the thermal data in the hard disk, and the data information of the corresponding location in the hard disk can be directly obtained according to the hard disk storage location information of the thermal data stored in the data management area 10.
The data reading method provided by the embodiment of the invention comprises the steps of firstly obtaining data to be read, judging whether the attribute of the data to be read is thermal data or not, and directly reading the thermal data by directly obtaining the storage position information of a hard disk of the thermal data in a data management area when the data to be read is the thermal data; when the data is judged to be cold data, the cold data can be directly obtained through obtaining the information of the storage position of the optical disk library of the cold data in the data management area, reading and processing are carried out, and the data can be directly obtained through different position information of the data by combining the reading mode of the data and the access frequency of the data to be read, so that the data reading speed is improved, the data processing is facilitated, and the consumption of storage media such as a magnetic disk is reduced.
As an optional implementation manner of the present application, the method for reading data provided in the embodiment of the present invention further includes:
step S25: when the data is cold data, the optical disc library storing location information of the cold data in the data management area is acquired. In this embodiment, when the data to be read is cold data, the cold data is stored in the optical disc library, and the optical disc library storage location information of the cold data is stored in the data management area according to the data storage method described in any of the above embodiments, and at this time, the data information of the cold data is acquired by acquiring the storage location information of the cold data in the data management area.
Step S26: acquiring cold data according to the storage position information of the optical disk library;
step S27: and reading the obtained cold data into a hard disk, and reading and processing.
In the data reading method provided by the embodiment of the present invention, in the cold data storage optical disc library, the storage location information of the optical disc library of the cold data is stored in the data management area, and when it is determined that the data to be read is the cold data, the data information of the corresponding location of the optical disc library may be directly obtained according to the storage location information of the optical disc library, or after the data information of the cold data is obtained, the cold data may be read into the hard disc to process the data. The reading speed of the mechanical arm of the optical disk library is low, so that the data in the optical disk library is buffered in the hard disk, and then the data is read and processed, so that the reading speed of the data is increased, and the reading efficiency of the data is also improved.
As an optional implementation manner of the present application, the method for reading data provided in the embodiment of the present invention further includes:
judging whether the current access frequency of the hot data stored in the hard disk is lower than the preset access frequency or not according to the preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, cleaning the hot data in the hard disk, feeding back cleaning information of the hot data to the data management area, modifying attribute information of the hot data, and deleting the hard disk storage position information of the hot data in the data management area.
According to the data reading method provided by the embodiment of the invention, the access frequency of the hot data stored in the hard disk is judged regularly, and when the access frequency becomes lower and is lower than the preset access frequency, the data is cleaned. And the attribute information of the data is modified in the data management area, the position information of the hard disk corresponding to the data in the data management area is deleted, and the data which does not accord with the hot data standard in the hard disk is deleted, so that the purpose of saving the hard disk space is achieved, and the cost of data storage is reduced.
Example 3
The embodiment of the present invention provides a data storage device, which can be applied to a specific application scenario where large-scale organizations such as banks, governments, and companies need to store data with different access frequencies, as shown in fig. 4, the data storage device includes:
the first obtaining module 31 is configured to obtain data to be stored, where an attribute of the data to be stored is default to cold data, where the attribute of the data includes hot data or cold data, and is configured to store attribute information of the data to be stored in a data management area, where the data management area is a storage space arranged in a memory, and details of implementation may be referred to related description of step S11 in the foregoing method embodiment.
The first determining module 32 is configured to determine whether an access frequency of the data to be stored is greater than or equal to a preset access frequency, where the data with the access frequency greater than or equal to the preset access frequency is hot data, and the data with the access frequency less than the preset access frequency is cold data, and details of implementation may be referred to related description of step S12 in the foregoing method embodiment.
The first storage module 33 is configured to, when the data to be stored is hot data, store the hot data in a hard disk, and modify attribute information of the data to be stored in the data management area, for details, see the related description of step S13 in the foregoing method embodiment.
The second obtaining module 34 is configured to obtain a storage location of the thermal data in the hard disk, and store information of the storage location of the thermal data in the hard disk into the data management area, and details of implementation may be referred to related description of step S14 of the foregoing method embodiment.
The second storage module 35 is configured to write all data to be stored into the optical disc library, and details of implementation may be referred to in the related description of step S15 of the foregoing method embodiment.
The third obtaining module 36 is configured to obtain storage locations of all data to be stored in the optical disc library, and store information of the storage locations of all data to be stored in the optical disc library in the data management area, for details of implementation, refer to the related description of step S16 in the foregoing method embodiment.
According to the data storage method provided by the embodiment of the invention, the attribute information of the data to be stored is judged, namely the access frequency of the data to be stored is judged, the attribute judgment result information of the data to be stored is stored in the data management area, when the access frequency is less than the preset access frequency, the data is determined to be cold data, and the attribute information of the cold data is stored in the data management area, so that the problem that the data can be stored only after being magnetically/electrically buffered in the prior art is solved, and different storage paths can be selected to store the data by combining with the new attribute information of the data, so that the data storage step is simplified, the data storage cost is saved, and the read-write times of storage media such as a magnetic disk are reduced.
Example 4
The embodiment of the present invention provides a data reading apparatus, which can be applied to a specific application scenario where a large organization such as a bank, a government, a company, etc. needs to read stored data, as shown in fig. 5, the data reading apparatus includes:
the fourth obtaining module 41 is configured to obtain data to be read, and details of implementation may be referred to in the related description of step S21 of the foregoing method embodiment.
The second judging module 42 is configured to judge whether an attribute of the data to be read is hot data; the thermal data refers to data stored in the data management area through the data storage device as in embodiment 3, and the detailed implementation can be referred to the related description of step S22 of the above method embodiment.
The fifth obtaining module 43 is configured to, when the data to be read is hot data, directly obtain the hard disk storage location information of the hot data in the data management area, and the detailed implementation contents may be referred to the related description of step S23 of the foregoing method embodiment.
The reading module 44 is configured to read the thermal data according to the hard disk storage location information, and the detailed implementation contents may be referred to in the related description of step S24 of the foregoing method embodiment.
The data reading method provided by the embodiment of the invention comprises the steps of firstly obtaining data to be read, judging whether the attribute of the data to be read is thermal data or not, and directly reading the thermal data by directly obtaining the storage position information of a hard disk of the thermal data in a data management area when the data to be read is the thermal data; when the data is judged to be cold data, the cold data can be directly obtained through obtaining the information of the storage position of the optical disk library of the cold data in the data management area, reading and processing are carried out, and the data can be directly obtained through different position information of the data by combining the reading mode of the data and the access frequency of the data to be read, so that the data reading speed is improved, the data processing is facilitated, and the consumption of storage media such as a magnetic disk is reduced.
Example 5
The present embodiment provides a control apparatus, as shown in fig. 6, including:
the first communication module 511: the data transmission device is used for transmitting data and transmitting data to be stored or data to be read; the first communication module can be a Bluetooth module and a Wi-Fi module, and then communication is carried out through a set wireless communication protocol.
The first controller 512: connected to the first communication module 511, as shown in fig. 7, includes: at least one processor 61; and a memory 62 communicatively coupled to the at least one processor 61; the memory 62 stores instructions executable by the at least one processor 61, and when receiving data information, the at least one processor 61 is enabled to execute a data storage method shown in fig. 2 or a data reading method shown in fig. 3, in fig. 7, taking one processor as an example, the processor 61 and the memory 62 are connected through the bus 60, in this embodiment, the first communication module may be a wireless communication module, for example, a bluetooth module, a Wi-Fi module, or the like, or may be a wired communication module. The transmission between the first controller 512 and the first communication module 511 is a wireless transmission.
The processor 61 may be a Central Processing Unit (CPU). The Processor 61 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 62 is a non-transitory computer readable storage medium, and can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the data storage method or the data reading method in the embodiments of the present application. The processor 61 executes various functional applications of the server and data processing, i.e., realizes the storage method of data or the reading method of data in the above-described embodiments, by executing the non-transitory software programs, instructions, and modules stored in the memory 62.
The memory 62 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of a processing device operated by the server, and the like. Further, the memory 62 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 62 may optionally include memory located remotely from the processor 61, which may be connected to a network connection device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
One or more modules are stored in the memory 62 and, when executed by the one or more processors 61, perform the method described in any of the above embodiments.
The embodiment of the present invention further provides a non-transitory computer readable medium, where the non-transitory computer readable storage medium stores a computer instruction, and the computer instruction is used to enable a computer to execute a data storage method or a data reading method described in any of the above embodiments, where the storage medium may be a magnetic Disk, an optical Disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a flash Memory (FlashMemory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid-State Drive (Solid-State Drive, SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
Claims (11)
1. A method for storing data, comprising the steps of:
acquiring data to be stored, wherein the attribute of the data to be stored is defaulted to cold data, and the attribute of the data comprises hot data or cold data;
storing the attribute information of the data to be stored into a data management area, wherein the data management area is a storage space arranged in a memory;
judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency or not, wherein the data with the access frequency greater than or equal to the preset access frequency are hot data, and the data with the access frequency less than the preset access frequency are cold data;
when the data to be stored is thermal data, storing the thermal data into a hard disk, and modifying the attribute information of the data to be stored in the data management area;
acquiring the storage position of the thermal data in a hard disk, and storing the storage position information of the thermal data in the hard disk into the data management area;
writing all the data to be stored into an optical disk library;
and acquiring the storage positions of all the data to be stored in the optical disc library, and storing the storage position information of all the data to be stored in the optical disc library into the data management area.
2. The data storage method according to claim 1, further comprising, after acquiring the data to be stored: and marking the data to be stored, and storing the identification information of the data to be stored into the data management area, wherein the identification information of the data to be stored is unique.
3. The data storage method according to claim 2, further comprising:
judging whether the current access frequency of the hot data stored in the hard disk is lower than a preset access frequency or not according to a preset time interval;
when the current access frequency of the hot data is lower than the preset access frequency, cleaning the hot data in the hard disk, feeding back cleaning information of the hot data to a data management area, modifying attribute information of the hot data, and deleting the hard disk storage position information of the hot data in the data management area.
4. A method for reading data, comprising:
acquiring data to be read;
judging whether the attribute of the data to be read is hot data or not; the thermal data refers to data stored in a hard disk by the data storage method according to any one of claims 1 to 3;
when the data to be read is hot data, directly acquiring the hard disk storage position information of the hot data in the data management area;
and reading the thermal data according to the storage position information of the hard disk.
5. The data reading method according to claim 4, further comprising:
when the data is cold data, acquiring storage position information of an optical disk library of the cold data in a data management area;
acquiring the cold data according to the storage position information of the optical disk library;
and reading the obtained cold data into a hard disk, and reading and processing.
6. The data reading method according to claim 5, further comprising:
judging whether the current access frequency of the hot data stored in the hard disk is lower than a preset access frequency or not according to a preset time interval;
when the current access frequency of the hot data is lower than the preset access frequency, cleaning the hot data in the hard disk, feeding back cleaning information of the hot data to a data management area, modifying attribute information of the hot data, and deleting the hard disk storage position information of the hot data in the data management area.
7. A data storage device, comprising
The device comprises a first acquisition module, a second acquisition module and a data management module, wherein the first acquisition module is used for acquiring data to be stored, the attribute of the data to be stored is default to be cold data, the attribute of the data comprises hot data or cold data, and the first acquisition module is also used for storing the attribute information of the data to be stored into a data management area, and the data management area is a storage space arranged in a memory;
the first judging module is used for judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency or not, the data with the access frequency greater than or equal to the preset access frequency are hot data, and the data with the access frequency less than the preset access frequency are cold data;
the first storage module is used for storing the hot data into a hard disk and modifying the attribute information of the data to be stored in the data management area when the data to be stored is the hot data;
the second acquisition module is used for acquiring the storage position of the thermal data in the hard disk and storing the storage position information of the thermal data in the hard disk into the data management area;
the second storage module is used for writing all the data to be stored into an optical disk library;
and the third acquisition module is used for acquiring the storage positions of all the data to be stored in the optical disc library and storing the storage position information of all the data to be stored in the optical disc library into the data management area.
8. An apparatus for reading data, comprising:
the fourth acquisition module is used for acquiring data to be read;
the second judgment module is used for judging whether the attribute of the data to be read is hot data or not; the thermal data is data stored in a hard disk by the data storage device according to claim 7;
a fifth obtaining module, configured to directly obtain, when the data to be read is hot data, location information of a hard disk storage of the hot data in the data management area;
and the reading module is used for reading the thermal data according to the storage position information of the hard disk.
9. A system for storing data, comprising:
at least one control device for executing the method of storing data according to any one of claims 1 to 3, storing the received data to be stored.
10. A system for reading data, comprising:
at least one control device for executing a method for reading data according to any one of claims 4 to 6, for reading data.
11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of a method for storing data according to any one of claims 1 to 3 or a method for reading data according to any one of claims 4 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911310870.8A CN111124303B (en) | 2019-12-18 | 2019-12-18 | Data storage method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911310870.8A CN111124303B (en) | 2019-12-18 | 2019-12-18 | Data storage method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111124303A true CN111124303A (en) | 2020-05-08 |
| CN111124303B CN111124303B (en) | 2024-06-07 |
Family
ID=70499742
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911310870.8A Active CN111124303B (en) | 2019-12-18 | 2019-12-18 | Data storage method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111124303B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111782148A (en) * | 2020-06-30 | 2020-10-16 | 中国工商银行股份有限公司 | Data storage control method and device, electronic equipment and storage medium |
| CN112379842A (en) * | 2020-11-18 | 2021-02-19 | 深圳安捷丽新技术有限公司 | Method and device for predicting cold and hot properties of data |
| CN114816232A (en) * | 2021-07-27 | 2022-07-29 | 云南省地质环境监测院(云南省环境地质研究院) | Method and device for efficiently accessing geological disaster big data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850358A (en) * | 2015-05-26 | 2015-08-19 | 华中科技大学 | Magnetic-optical-electric hybrid storage system and data acquisition and storage method thereof |
| CN106547474A (en) * | 2015-09-22 | 2017-03-29 | 辛力彬 | A kind of box mixing data storage center |
| CN108958663A (en) * | 2018-07-23 | 2018-12-07 | 南京叠嘉信息科技有限公司 | Data read-write method based on hard disk and CD mixing storage system |
| US20190265895A1 (en) * | 2018-02-27 | 2019-08-29 | Cisco Technology, Inc. | Method and Device for Improving File System Write Bandwidth through Hard Disk Track Management |
-
2019
- 2019-12-18 CN CN201911310870.8A patent/CN111124303B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850358A (en) * | 2015-05-26 | 2015-08-19 | 华中科技大学 | Magnetic-optical-electric hybrid storage system and data acquisition and storage method thereof |
| CN106547474A (en) * | 2015-09-22 | 2017-03-29 | 辛力彬 | A kind of box mixing data storage center |
| US20190265895A1 (en) * | 2018-02-27 | 2019-08-29 | Cisco Technology, Inc. | Method and Device for Improving File System Write Bandwidth through Hard Disk Track Management |
| CN108958663A (en) * | 2018-07-23 | 2018-12-07 | 南京叠嘉信息科技有限公司 | Data read-write method based on hard disk and CD mixing storage system |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111782148A (en) * | 2020-06-30 | 2020-10-16 | 中国工商银行股份有限公司 | Data storage control method and device, electronic equipment and storage medium |
| CN111782148B (en) * | 2020-06-30 | 2023-09-05 | 中国工商银行股份有限公司 | Data storage control method and device, electronic equipment and storage medium |
| CN112379842A (en) * | 2020-11-18 | 2021-02-19 | 深圳安捷丽新技术有限公司 | Method and device for predicting cold and hot properties of data |
| CN114816232A (en) * | 2021-07-27 | 2022-07-29 | 云南省地质环境监测院(云南省环境地质研究院) | Method and device for efficiently accessing geological disaster big data |
| CN114816232B (en) * | 2021-07-27 | 2022-11-08 | 云南省地质环境监测院(云南省环境地质研究院) | Method and device for efficiently accessing geological disaster big data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111124303B (en) | 2024-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102134694B1 (en) | Method, computer-readable storage media and storage controller module for managing storage regions on a storage device | |
| CN111737265B (en) | Block data access method, block data storage method and device | |
| CN111124303A (en) | Data storage method, device and system | |
| CN104516471A (en) | Method and device for managing power supply of storage system | |
| CN103430174A (en) | Selecting storage locations for storing data based on storage location attributes and data usage statistics | |
| CN107291399B (en) | Back-end storage method, device and system based on SPDK | |
| CN112684987B (en) | Data classified storage method and device based on double-core intelligent ammeter | |
| CN110727404A (en) | Data deduplication method and device based on storage end and storage medium | |
| CN101114508A (en) | Information recording apparatus and control method thereof | |
| CN101706822A (en) | Method and device for improving speed of mounting journal file system | |
| US9804781B2 (en) | Storage media performance management | |
| CN102622315B (en) | Method, device and system for controlling disc visiting | |
| CN111625203A (en) | Method, system, device and medium for hierarchical storage | |
| US20170329553A1 (en) | Storage control device, storage system, and computer-readable recording medium | |
| CN114153394A (en) | Data storage management method, device and equipment | |
| JP2016103304A (en) | Data archive system | |
| CN105242985B (en) | Data recovery method and device | |
| US7200771B2 (en) | Relocation batch processing for disk drives | |
| CN114816262B (en) | Method, system, equipment and medium for balancing data writing of cache disk | |
| US11682428B2 (en) | Hard disk format conversion method and apparatus, and storage device | |
| CN206628271U (en) | CD server switch controlling device and optical-disk type data center | |
| CN116931833A (en) | Management system of disk box | |
| CN115562582A (en) | Data migration method based on blue-ray disc library, server and blue-ray disc library | |
| EP2306462B1 (en) | Semiconductor memory device and its early-warning system and method | |
| US10175907B2 (en) | Storage device, file writing method, and non-transitory computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |