CN118466855B - Memory chip and data compression method thereof - Google Patents
Memory chip and data compression method thereof Download PDFInfo
- Publication number
- CN118466855B CN118466855B CN202410917451.5A CN202410917451A CN118466855B CN 118466855 B CN118466855 B CN 118466855B CN 202410917451 A CN202410917451 A CN 202410917451A CN 118466855 B CN118466855 B CN 118466855B
- Authority
- CN
- China
- Prior art keywords
- data
- stored
- template
- repeated
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000013144 data compression Methods 0.000 title claims abstract description 21
- 238000012216 screening Methods 0.000 claims abstract description 6
- 238000004891 communication Methods 0.000 claims abstract description 5
- 238000012545 processing Methods 0.000 claims description 25
- 230000015654 memory Effects 0.000 claims description 19
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000013500 data storage Methods 0.000 description 6
- 238000007906 compression Methods 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 238000001914 filtration Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002035 prolonged effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000005299 abrasion Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a memory chip and a data compression method thereof, comprising the following steps: the main controller is in communication connection with the host computer and is used for acquiring a data set to be stored; the data comparison module is used for comparing a certain data to be stored in the data set to be stored with other data to be stored, comparing the certain data to be stored with template data in the storage chip, and counting the repetition times of the certain data to be stored when the certain data to be stored is repeated with the other data to be stored or the template data; sorting and screening a plurality of data to be stored according to the repetition times to obtain template data; the data template module is used for storing the template data and acquiring a target storage physical address of the template data; the data comparison module is also used for erasing other data except the template data in the target data set, so that the storage space utilization rate of the storage chip can be improved.
Description
Technical Field
The present invention relates to the field of electronic memory technologies, and in particular, to a memory chip and a data compression method thereof.
Background
With the development of the age, various APP and video occupy more and more memory space, and then the requirement for data compression is higher and higher. The compression capability of the upper layer software has been developed to a certain stage, and then the compression capability of the bottom layer memory chip (Universal Flash Storage, UFS) becomes an important research direction in the future.
UFS memory chips are a complex integrated memory system whose internal architecture is designed to improve data transfer rates and memory efficiency. However, the existing data compression method has poor compression effect on the UFS memory chip, and the space occupied by the compressed data in the UFS memory chip is large, so that the storage space utilization rate of the UFS memory chip is reduced. Therefore, there is a need for improvement.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a memory chip and a data compression method thereof, which improves the memory space utilization of the memory chip.
In order to solve the technical problems, the invention is realized by the following technical scheme:
the present invention provides a memory chip, comprising:
the main controller is in communication connection with the host and is used for acquiring a data set to be stored of the host, and the main controller also comprises a data comparison module and a data template module;
The data comparison module is used for comparing each piece of data to be stored in the data set to be stored with other pieces of data to be stored respectively, and comparing each piece of data to be stored with template data in a storage chip respectively, and counting the repetition times of a certain piece of data to be stored when the certain piece of data to be stored in the data set to be stored is repeated with the other pieces of data to be stored or the template data, and counting the repetition times of each piece of data to be stored in the data set to be stored;
the data comparison module is also used for sorting each data to be stored in the data set to be stored according to the repetition times to obtain a target data set;
the data comparison module is also used for screening the data to be stored in the target data set to obtain template data;
The data template module is used for storing the template data and acquiring a target storage physical address of the template data;
the data comparison module is also used for erasing other data except the template data in the target data set.
In an embodiment of the present invention, the memory chip further includes a memory area for storing other data to be stored in the set of data to be stored that is not repeated with each data to be stored, and the memory area is further configured to store data to be stored in the set of data to be stored that is not repeated with the template data.
In an embodiment of the present invention, the master controller further includes a data traversing module, configured to perform data traversing processing on the data set to be stored.
In an embodiment of the present invention, when the data comparison module determines that the data set to be stored is received, the action performed is: judging whether each data to be stored in the data set to be stored is repeated with other data to be stored or the template data;
If a certain data to be stored in the data set to be stored is repeated with other data to be stored or the template data, counting the repeated times of the certain data to be stored, and counting the repeated times of each data to be stored in the data set to be stored;
If the data to be stored in the data set to be stored is not repeated with the other data to be stored and the template data, the data to be stored is directly stored in a storage area, and whether the other data to be stored in the data set to be stored is repeated with the other data to be stored or the template data is repeatedly judged until the other data to be stored in the data set to be stored is repeated with the other data to be stored or the template data.
In an embodiment of the present invention, when the data comparison module performs the sorting processing on a plurality of data to be stored according to the repetition number, the actions performed are:
Judging whether each data to be stored in the data set to be stored is repeated with the template data or not;
if the data to be stored in the data set to be stored is repeated with the template data, carrying out numerical value addition processing on the total repetition number of the data to be stored to obtain target repetition number, and carrying out sorting processing on each data to be stored in the data set to be stored according to the target repetition number and the repetition number of other data to be stored to obtain a target data set;
And if each piece of data to be stored in the data set to be stored is not repeated with the template data, sorting each piece of data to be stored in the data set to be stored according to the repetition times, and obtaining a target data set.
In an embodiment of the present invention, the data comparison module determines that the actions performed after the template data is acquired are: modifying physical addresses of other data except the template data in the target data set into the target storage physical addresses;
And performing data erasure processing on other data except the template data in the target data set.
The invention also provides a data compression method of the memory chip, which comprises the following steps:
acquiring a data set to be stored;
Comparing each data to be stored in the data set to be stored with other data to be stored respectively, and comparing each data to be stored with template data in a storage chip respectively, counting the repetition number of a certain data to be stored when the certain data to be stored in the data set to be stored is repeated with the other data to be stored or the template data, and counting the repetition number of each data to be stored in the data set to be stored;
sequencing each data to be stored in the data set to be stored according to the repetition times to obtain a target data set;
screening the data to be stored in the target data set to obtain template data;
storing the template data to obtain a target storage physical address of the template data;
and erasing other data except the template data in the target data set.
In an embodiment of the present invention, the step of comparing one data to be stored in the data set to be stored with other data to be stored, and comparing each data to be stored with template data in a memory chip respectively includes:
judging whether each data to be stored in the data set to be stored is repeated with other data to be stored or the template data;
If a certain data to be stored in the data set to be stored is repeated with other data to be stored or the template data, counting the repeated times of the certain data to be stored, and counting the repeated times of each data to be stored in the data set to be stored;
if the data to be stored in the data set to be stored is not repeated with the other data to be stored and the template data, the data to be stored is directly stored in a storage area, and whether the other data to be stored in the data set to be stored is repeated with the other data to be stored or the template data is repeatedly judged until the other data to be stored in the data set to be stored is repeated with the other data to be stored or the template data.
In an embodiment of the present invention, the step of sorting each data to be stored in the data set to be stored according to the repetition number includes:
Judging whether each data to be stored in the data set to be stored is repeated with the template data or not;
if the data to be stored in the data set to be stored is repeated with the template data, carrying out numerical value addition processing on the total repetition number of the data to be stored to obtain target repetition number, and carrying out sorting processing on each data to be stored in the data set to be stored according to the target repetition number and the repetition number of other data to be stored to obtain a target data set;
And if each piece of data to be stored in the data set to be stored is not repeated with the template data, sorting each piece of data to be stored in the data set to be stored according to the repetition times, and obtaining a target data set.
In an embodiment of the present invention, the step of erasing data other than the template data in the target data set includes:
modifying physical addresses of other data except the template data in the target data set into the target storage physical addresses;
And performing data erasure processing on other data except the template data in the target data set.
As described above, the data comparison module is arranged in the master controller to compare a certain data to be stored in the traversed data set to other data to be stored, and compare the certain data to be stored with the template data in the memory chip, so as to count the repetition number of the certain data to be stored, sort the plurality of data to be stored according to the repetition number to obtain a target data set, screen the data to be stored in the target data set to obtain the template data, erase other data in the target data set according to the target storage physical address of the template data, and realize data duplication in the memory chip, avoid repeated storage of the data in the memory chip, thereby realizing data compression in the memory chip, improving the data compression effect, saving the memory space in the memory chip, and improving the space utilization rate of the memory chip.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a memory chip according to the present invention;
FIG. 2 is a flow chart of a method for compressing data of a memory chip according to the present invention;
fig. 3 is a schematic flow chart of step S20 in fig. 2;
Fig. 4 is a schematic flow chart of step S30 in fig. 2;
Fig. 5 is a schematic flow chart of step S60 in fig. 2.
Description of element numbers:
100. A memory chip; 110. a master controller; 111. a data traversing module; 112. a data comparison module; 113. a data template module; 120. a storage area; 130. a host interface;
200. and a host.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the present invention provides a memory chip, which can be used for performing data compression processing on the memory chip 100 to reduce the occupied space of data to be stored in the memory chip 100, thereby improving the utilization rate of the memory space of the memory chip 100 and prolonging the service life of the memory chip 100.
Referring to fig. 1, the memory chip 100 may be a UFS chip (Universal Flash Storage), which is a memory chip with a high-performance interface standard and may be used in digital cameras, smart phones, tablet computers, and other various electronic devices. UFS chips can be designed to increase data transfer speed and efficiency while reducing power consumption. The UFS chip supports full duplex transmission based on a serial communication interface, and performs read and write operations at the same time, making it an ideal storage option for mobile devices and high performance electronic devices by providing higher data transmission speeds, improved energy efficiency, and increased reliability. Furthermore, the UFS standard supports command queuing techniques, which means that it can handle multiple instructions simultaneously, further improving performance. However, the memory chip 100 may be other types of memories as long as data storage can be realized.
Referring to fig. 1, further, the memory chip 100 may include, but is not limited to, a master 110, a memory area 120, and a host interface 130. The storage area 120 may be composed of a plurality of NAND flash memories, which may be used to store data. The master 110 may communicate with the host 200 through the host interface 130 to obtain a set of data to be stored and a data read-write request of the host 200. In addition, the master 110 may also be responsible for managing the NAND flash memory and handling the underlying tasks such as error detection and correction (ECC), wear leveling, defragmentation, etc.
Referring to fig. 1, further, the master 110 may include, but is not limited to, a data traversing module 111, a data comparing module 112, and a data template module 113. The data traversing module 111 may be communicatively connected to the data comparing module 112 to perform data traversing processing on all data to be stored in the set of data to be stored. The data traversal module 111 can ensure that each of the set of data to be stored within the master 110 can be accessed once and only once. The data traversal refers to a process of accessing each data to be stored in the data set to be stored by a systematic method, so as to find repeated data to be stored in the data set to be stored, and check the validity of the data to be stored. The data traversal process is typically automated by executing software logic within the master 110 of the memory chip 100 without the involvement of an external processor.
Referring to fig. 1, the data comparison module 112 may be configured to compare a certain data to be stored in the data set after the data traversing module 111 traverses the data set with other data to be stored, and compare the certain data to be stored with the template data in the memory chip 100, and count the repetition number of a certain data to be stored when the certain data to be stored is repeated with other data to be stored or the template data, so as to realize the duplication checking of all the data to be stored in the data set to be stored in the master 110.
Specifically, when the data comparison module 112 determines that the traversing data set to be stored is received, the action performed by the data comparison module 112 may be to determine whether a certain data to be stored in the data set to be stored is repeated with other data to be stored or template data. If the data to be stored is repeated with other data to be stored or template data, counting the repeated times of the data to be stored. If the data to be stored is not repeated with other data to be stored and the template data, the data to be stored is directly stored in the storage area 120, and whether the other data to be stored in the data set to be stored is repeated with other data to be stored or the template data is repeatedly judged until the data to be stored is repeated with other data to be stored or the template data. That is, the storage area 120 may be used to store other data to be stored in the data set that is not repeated with some data to be stored, and the storage area 120 is also used to store data to be stored in the data set that is not repeated with the template data.
Counting the repetition number of the certain data to be stored refers to judging whether the certain data to be stored in the data set to be stored is repeated with other traversed data to be stored or template data, and if so, counting and adding one to the certain data to be stored to record the repetition number of the certain data to be stored in the data set to be stored.
Referring to fig. 1, the data comparing module 112 is further configured to sort a plurality of data to be stored according to the repetition number, so as to obtain a target data set. Specifically, when the data comparison module 112 performs the sorting process on a plurality of data to be stored according to the repetition number, the action performed may be to determine whether the plurality of data to be stored is repeated with the template data. If the data to be stored and the template data are repeated, performing numerical value addition processing on the total repetition number of the data to be stored to obtain target repetition number, and performing sorting processing on a plurality of data to be stored according to the target repetition number and the repetition number of other data to be stored to obtain a target data set. And if the plurality of data to be stored and the template data are not repeated, sorting the plurality of data to be stored according to the repetition times to obtain a target data set. The purpose of determining whether the plurality of data to be stored is repeated with the template data is to facilitate sorting of the plurality of data to be stored according to the repetition number.
Further, the data comparison module 112 is further configured to perform a filtering process on the plurality of data to be stored in the target data set, so as to obtain template data. Specifically, according to the above steps, each data to be stored in the target data set is data with a certain number of repetitions, and the arrangement order of the data to be stored in the target data set may be determined from high to low according to the number of repetitions of the respective data to be stored. However, the arrangement order of the data to be stored in the target data group is not limited thereto, and may be determined from low to high according to the number of repetitions of the respective data to be stored.
Further, the number of template data may be determined according to the number of data templates set in the memory chip 100. For example, when n data templates are set in the memory chip 100, the number of data to be stored in the target data group is m, m is greater than or equal to n, and the arrangement order of the m data to be stored is determined according to the number of repetitions from high to low, the first n data to be stored may be represented as template data. When n data templates are set in the memory chip 100, the number of data to be stored in the target data set is m, m is smaller than n, and the arrangement order of the m data to be stored is determined according to the repetition number from high to low, all the data to be stored in the target data set can be used as template data.
However, without limitation, when there is a certain data to be stored in the target data set and template data preset in advance in the memory chip 100 are repeated, and the arrangement position of the certain data to be stored in the target data set is located in n data templates, the template data preset in advance may be continuously used as final template data. When some data to be stored in the target data set is repeated with the template data preset in advance in the memory chip 100, but the arrangement position of the some data to be stored in the target data set is beyond the n data templates, the template data preset in advance needs to be removed, so that space is saved for generating the subsequent template data.
Referring to fig. 1, for example, when the set of data to be stored includes 20 data to be stored, the 20 data to be stored may be divided into 5 groups, A1, A2, A3, A4, and A5, respectively. Wherein, A1 may include 6 data A1 to be stored, and all 6 data A1 to be stored are the same. There may be 5 pieces of data A2 to be stored in A2, and all of the 5 pieces of data A2 to be stored are the same. The A3 may include 4 data A3 to be stored therein, and the 4 data A3 to be stored are all the same. The A4 may include 3 data A4 to be stored therein, and all of the 3 data A4 to be stored are the same. The A5 may include 2 data to be stored A5, where the 2 data to be stored A5 are all the same, and a1, a2, a3, a4, and A5 are different from each other.
Thus, the target data set may be obtained, and the target data set may include 5 different data to be stored, which are a1, a2, a3, a4, and a5, respectively. When 5 data templates are provided in the memory chip 100, 5 different template data, a1, a2, a3, a4, and a5, respectively, can be acquired according to the target data set. However, without limitation, when only 2 data templates are provided in the memory chip 100, two template data, a1 and a2, respectively, may be obtained by filtering the target data set.
Referring to FIG. 1, a data template module 113 may be communicatively coupled to the data comparison module 112 for storing template data to obtain a target storage physical address for the template data. In addition, by setting the data template module 113 to be in communication connection with the data comparison module 112, the data comparison module 112 can also conveniently and directly call the template data to determine whether the data to be stored is the same or not when the main controller 110 continues to acquire the data to be stored of the host 200, so as to improve the data compression efficiency in the memory chip 100.
Referring to fig. 1, the data comparison module 112 is further configured to erase other data except the template data in the target data set, so as to delete the repeated data in the memory chip 100, thereby making room for the subsequent data storage, and realizing data compression in the memory chip 100. Specifically, after the data comparison module 112 determines that the template data is obtained, the action to be performed may be to modify the physical address of the data other than the template data in the target data set to the target storage physical address, and then perform the data erasure process on the data other than the template data in the target data set.
The modification of the physical address of other data to the target storage physical address is mainly used for realizing optimization and data compression of the data storage process. First, the physical addresses of the data in the target data group are all modified into the target storage physical address mapped by the logical address of the template data, so that preparation can be made for subsequent data erasure, and the data can be erased intensively. In addition, by modifying all the physical addresses of other data in each other target data group into the target storage physical address corresponding to each template data, the other physical addresses can be left, so that the inherent space in the memory chip 100 is saved, and the utilization rate of the internal space of the memory chip 100 is improved.
Referring to fig. 1, further, the main purpose of performing data erasure processing on other data except for template data in the target data set is to optimize the memory space inside the memory chip 100. By erasing the data that has been represented or compressed by the template data, the amount of data stored in the actual memory chip 100 can be reduced, thereby improving the utilization of the memory space. In addition, by erasing the data, the write operation from the main controller 110 to the storage area 120 can be reduced, the abrasion to the storage area 120 can be reduced, and the service life of the memory chip 100 can be further prolonged.
Referring to fig. 1, in an embodiment of the present invention, when the memory chip 100 stores template data, the logical address of the template data is X0, the corresponding target storage physical address is P0, and the acquired set of data to be stored of the host 200 includes m pieces of data to be stored, and the corresponding logical addresses are X1 to Xm, the data traversing module 111 may first perform traversing processing on the data to be stored, the logical addresses of which are X1 to Xm, respectively, and call the data comparing module 112 to sequentially compare the data to be stored, the logical addresses of which are X1 to Xm, with the template data, the logical address of which is X0, so as to determine whether the data to be stored, the logical addresses of which are X1 to Xm, respectively, are identical with the template data, the logical address of which is X0. If the data to be stored with the logic addresses of X1-Xm are the same as the template data with the logic address of X0, the repetition times of the data to be stored are recorded, and when one data to be stored is found to be the same as the template data, the count of the data is increased by one until all the judgment is completed. To obtain a set of data to be stored identical to the template data and to represent it as a first set of repeating data.
Further, when the data to be stored in the data sets X1 to Xm are different from each other except for the first repeated data set, the first repeated data set may be represented as the first target data set. Because the template data already exist, the physical addresses of all data in the first target data set can be directly modified into the same physical addresses as the template data, and the first target data set is subjected to data erasing processing, so that the storage space in the memory chip 100 is saved, and the space utilization rate is improved. In addition, other data to be stored, which are different from each other in the data sets X1 to Xm, need to be stored in the storage area 120.
However, the method is not limited thereto, and when there are a plurality of to-be-stored data identical to other to-be-stored data in addition to a plurality of to-be-stored data identical to the template data in the to-be-stored data sets X1 to Xm, the number of repetitions of the to-be-stored data may be counted, and the plurality of to-be-stored data may be sorted to obtain the second target data set.
Further, if two template data can be stored in the memory chip 100, a filtering process can be performed on a certain data to be stored corresponding to each of the multiple times of repetition of certain data to be stored in the second target data set according to the times of repetition of certain data to be stored in the second target data set, so as to represent a certain data to be stored with the largest times of repetition in the second target data set as second template data, and the second template data is stored in the data template module 113, so as to obtain a corresponding second target storage physical address. Finally, the data comparison module 112 may modify all physical addresses of other data except the second template data in the second target data set into second target storage physical addresses, and erase other data, so as to store only the second template data in the second target data set, thereby not only realizing data storage, but also saving space inside the memory chip 100 and improving space utilization in the memory chip 100. In addition, other data to be stored, which are different from each other in the data sets X1 to Xm, need to be stored in the storage area 120.
It should be further noted that, if only one template data can be stored in the memory chip 100, the total number of repetitions of the data to be stored in the first repeated data set is added by one and then compared with the maximum number of repetitions in the second target data set, and if the number of repetitions of the data in the first repeated data set is greater than or equal to the maximum number of repetitions of the data to be stored in the second target data set after adding by one, the first repeated data set is required to be represented as the first target data set. Because the template data already exists, the data comparison module 112 can be utilized to modify the physical addresses of all the data in the first target data set into the same physical addresses as the template data, and perform data erasure processing on the first target data set, so as to save the storage space inside the memory chip 100 and improve the space utilization rate. In addition, a second target data set and other different data to be stored are required to be stored in the storage area 120.
It should be further noted that, if the total number of repetitions of the data to be stored in the first repeated data set is less than the maximum number of repetitions of the data to be stored in the second target data set after adding one, a certain data to be stored corresponding to the maximum number of repetitions in the second target data set may be represented as second template data, and the second template data may be stored in the data template module 113 to obtain a corresponding second target storage physical address, and the above steps are repeated to implement compression of the second repeated data. In addition, the first repeated data group and other different data to be stored are required to be stored in the storage area 120.
In the above scheme, in the data storage stage, a data comparison module is arranged in the master controller to compare a certain data to be stored in the traversed data set to be stored with other data to be stored, and compare the certain data to be stored with template data in the memory chip, so as to count the repetition number of the certain data to be stored, sort the plurality of data to be stored according to the repetition number to obtain a target data set, screen the data to be stored in the target data set to obtain template data, erase other data in the target data set according to the target storage physical address of the template data, so that the data in the memory chip can be removed, the memory chip is prevented from repeatedly storing the data, the memory space in the memory chip is saved, the space utilization rate of the memory chip is improved, and the service life of the memory chip is further prolonged.
Referring to fig. 1 and fig. 2, the present invention further provides a data compression method of a memory chip, which can be applied to the memory chip 100 described above, so as to implement data compression in the data storage process of the memory chip 100 by a data deduplication method. The data compression method of the memory chip may include steps S10 to S60, which will be described in detail below.
Step S10, acquiring a data set to be stored.
And step S20, comparing a certain data to be stored in the data set to be stored with other data to be stored, comparing the certain data to be stored with the template data in the memory chip, and counting the repetition times of the certain data to be stored when the certain data to be stored is repeated with the other data to be stored or the template data.
And step S30, sorting a plurality of data to be stored according to the repetition times to obtain a target data set.
And S40, screening the data to be stored in the target data set to obtain template data.
And step S50, storing the template data to obtain a target storage physical address of the template data.
And S60, erasing other data except the template data in the target data group.
Referring to fig. 3, in one embodiment of the present invention, when step S20 is performed, specifically, step S20 may include steps S21 to S23, which are described in detail below.
Step S21, judging whether one piece of data to be stored in the data set to be stored is repeated with other pieces of data to be stored or template data.
Step S22, if the data to be stored is repeated with other data to be stored or template data, counting the repeated times of the data to be stored.
And S23, if the data to be stored is not repeated with other data to be stored and the template data, directly storing the data to be stored into the storage area, and repeatedly judging whether the other data to be stored in the data set to be stored is repeated with the other data to be stored or the template data until the other data to be stored is repeated with the other data to be stored or the template data.
Referring to fig. 4, in an embodiment of the present invention, when step S30 is performed, specifically, step S30 may include steps S31 to S32, which are described in detail below.
Step S31, judging whether a plurality of data to be stored are repeated with the template data.
And step S32, if the data to be stored and the template data are repeated, adding a value to the total repetition number of the data to be stored to obtain a target repetition number, and sorting the plurality of data to be stored according to the target repetition number and the repetition number of other data to be stored to obtain a target data set.
And step S33, if the plurality of data to be stored and the template data are not repeated, sorting the plurality of data to be stored according to the repetition times to obtain a target data set.
Referring to fig. 5, in one embodiment of the present invention, when step S60 is performed, specifically, step S60 may include steps S61 to S62, which are described in detail below.
And S51, modifying the physical addresses of other data to be stored except the template data in the target data set into target storage physical addresses.
Step S52, data erasing processing is carried out on other data except the template data in the target data group.
In the description of the present specification, the descriptions of the terms "present embodiment," "example," "specific example," and the like, mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The embodiments of the invention disclosed above are intended only to help illustrate the invention. The examples are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. The invention is limited only by the claims and the full scope and equivalents thereof.
Claims (10)
1. A memory chip, the memory chip comprising:
the main controller is in communication connection with the host and is used for acquiring a data set to be stored of the host, and the main controller also comprises a data comparison module and a data template module;
the data comparison module is used for comparing each piece of data to be stored in the data set to be stored with other pieces of data to be stored respectively, and comparing each piece of data to be stored with template data in a storage chip respectively, and counting the repetition times of a certain piece of data to be stored when the certain piece of data to be stored in the data set to be stored is repeated with the other pieces of data to be stored or the template data, and counting the repetition times of each piece of data to be stored in the data set to be stored;
the data comparison module is further used for sorting each data to be stored in the data set to be stored according to the repetition times to obtain a target data set;
the data comparison module is also used for screening the data to be stored in the target data set to obtain template data;
The data template module is used for storing the template data and acquiring a target storage physical address of the template data;
the data comparison module is also used for erasing other data except the template data in the target data set.
2. The memory chip of claim 1, further comprising a memory area to store other data in the set of data to be stored that is not repeated with each data to be stored, the memory area further to store data in the set of data to be stored that is not repeated with the template data.
3. The memory chip of claim 1, wherein the master further comprises a data traversing module to perform data traversing processing on the data set to be stored.
4. The memory chip of claim 1, wherein the data comparison module determines that the data set to be stored is received by performing the acts of: judging whether each data to be stored in the data set to be stored is repeated with other data to be stored or the template data;
If a certain data to be stored in the data set to be stored is repeated with other data to be stored or the template data, counting the repeated times of the certain data to be stored, and counting the repeated times of each data to be stored in the data set to be stored;
if the data to be stored in the data set to be stored is not repeated with the other data to be stored and the template data, the data to be stored is directly stored in a storage area, and whether the other data to be stored in the data set to be stored is repeated with the other data to be stored or the template data is repeatedly judged until the other data to be stored in the data set to be stored is repeated with the other data to be stored or the template data.
5. The memory chip of claim 1, wherein the data comparison module performs the following actions when ordering each of the data to be stored in the set of data to be stored according to the number of repetitions:
Judging whether each data to be stored in the data set to be stored is repeated with the template data or not;
if the data to be stored in the data set to be stored is repeated with the template data, carrying out numerical value addition processing on the total repetition number of the data to be stored to obtain target repetition number, and carrying out sorting processing on each data to be stored in the data set to be stored according to the target repetition number and the repetition number of other data to be stored to obtain a target data set;
And if each piece of data to be stored in the data set to be stored is not repeated with the template data, sorting each piece of data to be stored in the data set to be stored according to the repetition times, and obtaining a target data set.
6. The memory chip of claim 1, wherein the data comparison module determines that the template data is obtained, and performs the following actions: modifying physical addresses of other data except the template data in the target data set into the target storage physical addresses;
And performing data erasure processing on other data except the template data in the target data set.
7. A data compression method of a memory chip, comprising:
acquiring a data set to be stored;
Comparing each data to be stored in the data set to be stored with other data to be stored respectively, and comparing each data to be stored with template data in a storage chip respectively, counting the repetition number of a certain data to be stored when the certain data to be stored in the data set to be stored is repeated with the other data to be stored or the template data, and counting the repetition number of each data to be stored in the data set to be stored;
sequencing each data to be stored in the data set to be stored according to the repetition times to obtain a target data set;
screening the data to be stored in the target data set to obtain template data;
storing the template data to obtain a target storage physical address of the template data;
and erasing other data except the template data in the target data set.
8. The method of data compression of a memory chip of claim 7, wherein the comparing each of the data to be stored with the other data to be stored, respectively, and comparing each of the data to be stored with template data in the memory chip comprises:
judging whether each data to be stored in the data set to be stored is repeated with other data to be stored or the template data;
If a certain data to be stored in the data set to be stored is repeated with other data to be stored or the template data, counting the repeated times of the certain data to be stored, and counting the repeated times of each data to be stored in the data set to be stored;
if the data to be stored in the data set to be stored is not repeated with the other data to be stored and the template data, the data to be stored is directly stored in a storage area, and whether the other data to be stored in the data set to be stored is repeated with the other data to be stored or the template data is repeatedly judged until the other data to be stored in the data set to be stored is repeated with the other data to be stored or the template data.
9. The data compression method of a memory chip according to claim 7, wherein the step of sorting each of the data to be stored in the data set to be stored according to the number of repetitions includes:
Judging whether each data to be stored in the data set to be stored is repeated with the template data or not;
if the data to be stored in the data set to be stored is repeated with the template data, carrying out numerical value addition processing on the total repetition number of the data to be stored to obtain target repetition number, and carrying out sorting processing on each data to be stored in the data set to be stored according to the target repetition number and the repetition number of other data to be stored to obtain a target data set;
And if each piece of data to be stored in the data set to be stored is not repeated with the template data, sorting each piece of data to be stored in the data set to be stored according to the repetition times, and obtaining a target data set.
10. The method of data compression of a memory chip according to claim 7, wherein the step of performing erasure processing on the other data than the template data in the target data group includes:
modifying physical addresses of other data except the template data in the target data set into the target storage physical addresses;
And performing data erasure processing on other data except the template data in the target data set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410917451.5A CN118466855B (en) | 2024-07-10 | 2024-07-10 | Memory chip and data compression method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410917451.5A CN118466855B (en) | 2024-07-10 | 2024-07-10 | Memory chip and data compression method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118466855A CN118466855A (en) | 2024-08-09 |
| CN118466855B true CN118466855B (en) | 2024-09-27 |
Family
ID=92162280
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410917451.5A Active CN118466855B (en) | 2024-07-10 | 2024-07-10 | Memory chip and data compression method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118466855B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110795031A (en) * | 2019-10-17 | 2020-02-14 | 北京浪潮数据技术有限公司 | Data deduplication method, device and system based on full flash storage |
| CN110806984A (en) * | 2018-08-06 | 2020-02-18 | 爱思开海力士有限公司 | Apparatus and method for searching for valid data in memory system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2463920B (en) * | 2008-09-30 | 2012-08-22 | Cambridge Broadband Networks Ltd | Improved data compression |
| US9792069B2 (en) * | 2014-09-29 | 2017-10-17 | Western Digital Technologies, Inc. | Offline deduplication for solid-state storage devices |
| US20200210181A1 (en) * | 2018-12-29 | 2020-07-02 | Intel Corporation | Apparatuses, methods, and systems for vector element sorting instructions |
| US11321288B2 (en) * | 2020-08-05 | 2022-05-03 | Ocient Holdings LLC | Record deduplication in database systems |
| CN117331487A (en) * | 2022-06-24 | 2024-01-02 | 华为技术有限公司 | Data deduplication method and related system |
-
2024
- 2024-07-10 CN CN202410917451.5A patent/CN118466855B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110806984A (en) * | 2018-08-06 | 2020-02-18 | 爱思开海力士有限公司 | Apparatus and method for searching for valid data in memory system |
| CN110795031A (en) * | 2019-10-17 | 2020-02-14 | 北京浪潮数据技术有限公司 | Data deduplication method, device and system based on full flash storage |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118466855A (en) | 2024-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8055873B2 (en) | Data writing method for flash memory, and controller and system using the same | |
| US20130073798A1 (en) | Flash memory device and data management method | |
| CN106681654B (en) | Mapping table loading method and memory storage apparatus | |
| US8909895B2 (en) | Memory apparatus | |
| US9965194B2 (en) | Data writing method, memory control circuit unit and memory storage apparatus which performs data arrangement operation according to usage frequency of physical erasing unit of memory storage apparatus | |
| TWI615711B (en) | Data writing method, memory control circuit unit and memory storage apparatus | |
| CN107818808B (en) | Data writing method, memory control circuit unit and memory storage device | |
| US8429339B2 (en) | Storage device utilizing free pages in compressed blocks | |
| CN113885692A (en) | Memory efficiency optimization method, memory control circuit unit and memory device | |
| CN108733577A (en) | Storage management method, memorizer control circuit unit and memory storage apparatus | |
| CN110471612B (en) | Memory management method and memory controller | |
| US9880930B2 (en) | Method for operating controller and method for operating device including the same | |
| CN118466855B (en) | Memory chip and data compression method thereof | |
| CN103218308B (en) | Buffer storage supervisory method, Memory Controller and memorizer memory devices | |
| TWI571881B (en) | Valid data merging method, memory controller and memory storage apparatus | |
| CN112230849A (en) | Memory control method, memory storage device and memory controller | |
| CN110275668B (en) | Block management method, memory control circuit unit and memory storage device | |
| CN117111828A (en) | Hardware accelerated database ordering in solid state storage drives | |
| CN110008146B (en) | Data writing method, effective data identification method and memory storage device | |
| CN113094306A (en) | Effective data management method, memory storage device and memory controller | |
| TWI823792B (en) | Mapping table updating method, memory storage device and memory control circuit unit | |
| CN118331511B (en) | Memory management method and memory controller | |
| CN114115737B (en) | Data storage allocation method, memory storage device and control circuit unit | |
| US12093532B2 (en) | Data reading method, memory storage device, and memory control circuit unit | |
| US11620072B2 (en) | Memory management method, memory control circuit unit and memory storage apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |