CN111444219A - Distributed data processing method and device and electronic equipment - Google Patents
Distributed data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN111444219A CN111444219A CN202010237373.6A CN202010237373A CN111444219A CN 111444219 A CN111444219 A CN 111444219A CN 202010237373 A CN202010237373 A CN 202010237373A CN 111444219 A CN111444219 A CN 111444219A
- Authority
- CN
- China
- Prior art keywords
- data
- parameters
- parameter
- data block
- address information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000012545 processing Methods 0.000 claims description 13
- 238000012856 packing Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000010295 mobile communication Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24562—Pointer or reference processing operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the field of storage media, and discloses a data processing method and device based on distribution and an electronic device. The method comprises the following steps: acquiring parameters of queried data; retrieving in the first memory according to the parameters of the queried data to obtain address information corresponding to the parameters of the queried data; extracting data content corresponding to the parameters of the inquired data from the second memory according to the address information; the data content is output, so that the data operation period can be shortened, and the timeliness is strong.
Description
Technical Field
The present invention relates to the field of storage media, and in particular, to a data processing method and apparatus based on distributed mode, and an electronic device.
Background
When the device stores data, a large amount of historical record data (including consumption bills, operation records, alarm records, personnel information and the like) needs to be saved for practical application of the device. When effective data such as a certain day or a certain operation of a certain user is found from the data, the conventional method is to read each piece of data in the memory and compare the read data with the required data, if the read data are the same, the required data content is found, and if the read data are not the same, the data content is continuously inquired backwards until the required data content is found.
In view of the fact that nandflash is widely used as a low-cost large-capacity storage device when a storage device is selected at present, nandflash has the possibility of bad blocks when data are rapidly read or written, particularly, a sector erasing operation is required when data are written, the data operation period is long, and timeliness is poor when ultra-large data inquiry is carried out. And the operation efficiency of the equipment is even seriously influenced, and the production and the use are influenced.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a distributed data processing method, device and electronic device, which can shorten a data operation cycle and have high timeliness.
In a first aspect, an embodiment of the present invention provides a data processing method based on a distributed type, where the method includes:
acquiring parameters of queried data;
retrieving in a first memory according to the parameters of the queried data to obtain address information corresponding to the parameters of the queried data;
extracting data content corresponding to the parameters of the inquired data from a second memory according to the address information;
and outputting the data content.
In some embodiments, the obtaining parameters of the queried data includes:
and acquiring the type parameter of the data block and the characteristic parameter of the data block.
In some embodiments, the retrieving, in the first memory, according to the parameter of the queried data to obtain address information corresponding to the parameter of the queried data includes:
retrieving in a first memory according to the type parameter of the data block and the characteristic parameter of the data block;
if the characteristic parameters of the data block are not default;
outputting the address information of the data corresponding to the characteristic parameters of the data block.
In some embodiments, the retrieving, in the first memory, according to the parameter of the queried data to obtain address information corresponding to the parameter of the queried data includes:
retrieving in a first memory according to the type parameter of the data block and the characteristic parameter of the data block;
if the characteristic parameters of the data block are default;
and outputting the address information of the batch data corresponding to the type parameter of the data block.
In some embodiments, the method further comprises:
packing data to be stored into a data block, and storing the data block into the second memory;
acquiring type parameters of data blocks, characteristic parameters of the data blocks and address information of data corresponding to the type parameters of the data blocks and the characteristic parameters of the data blocks respectively;
and storing the type parameter of the data block, the characteristic parameter of the data block and address information of data corresponding to the type parameter of the data block and the characteristic parameter of the data block respectively into a data chain table area.
In some embodiments, the method further comprises:
acquiring a data structure of the data link table area;
and storing the data link table area to a corresponding storage space in a first memory according to the data structure of the data link table area.
In some embodiments, the method further comprises:
and pre-establishing a data link table area, wherein the data link table area is used for storing the type parameter of the data block, the characteristic parameter of the data block and address information of data corresponding to the type parameter of the data block and the characteristic parameter of the data block respectively.
In a second aspect, an embodiment of the present invention further provides a data processing apparatus based on distributed mode, where the apparatus includes:
the acquisition module is used for acquiring parameters of the inquired data;
the retrieval module is used for retrieving in the first memory according to the parameters of the queried data to obtain address information corresponding to the parameters of the queried data;
the extraction module is used for extracting data content corresponding to the parameters of the inquired data from a second memory according to the address information;
and the output module is used for outputting the data content.
In a third aspect, an embodiment of the present invention further provides an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-described distributed data processing-based method.
In a fourth aspect, the embodiments of the present invention also provide a non-volatile computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the processor is caused to execute the above-mentioned distributed data processing method.
Compared with the prior art, the invention has the beneficial effects that: different from the prior art, the distributed data processing method, the distributed data processing device and the electronic device in the embodiments of the present invention perform distributed storage on data in advance, so that when data is queried, a parameter of the queried data is obtained, and is retrieved from the first memory according to the parameter of the queried data, so as to obtain address information corresponding to the parameter of the queried data, and then data content corresponding to the parameter of the queried data is extracted from the second memory according to the address information and is output, and by retrieving the data parameter from the first memory and obtaining the corresponding address information, required data content is extracted from the second memory according to the address information, so that a data operation cycle can be shortened, and timeliness is strong.
Drawings
One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.
FIG. 1 is a flow diagram of a distributed based data processing method in one embodiment of the invention;
FIG. 2 is a flow chart illustrating the non-default of characteristic parameters of a data block according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating default of characteristic parameters of a data block according to an embodiment of the present invention;
FIG. 4 is a schematic flow diagram of data storage in one embodiment of the invention;
FIG. 5 is a schematic diagram of a structure in which data is stored sequentially or randomly according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a structure of a data link table area stored in a first memory according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a distributed data processing apparatus according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a hardware structure of an electronic device in an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that, if not conflicted, the various features of the embodiments of the invention may be combined with each other within the scope of protection of the invention. Additionally, while functional block divisions are performed in apparatus schematics, with logical sequences shown in flowcharts, in some cases, steps shown or described may be performed in sequences other than block divisions in apparatus or flowcharts. The terms "first", "second", "third", and the like used in the present invention do not limit data and execution order, but distinguish the same items or similar items having substantially the same function and action.
As shown in fig. 1, an embodiment of the present invention provides a data processing method based on distributed mode, where the method includes:
Specifically, the parameters of the queried data include a type parameter and a characteristic parameter, wherein the type parameter of the same data may have a plurality of different characteristic parameters, and when the user inputs the parameters of the queried data, the first memory obtains the parameters of the queried data.
And 104, retrieving in a first memory according to the parameters of the queried data to obtain address information corresponding to the parameters of the queried data.
In the embodiment of the present invention, the first memory is a charged erasable programmable read only memory, which is a read only memory that can be changed by a user, and can be erased and reprogrammed (rewritten) by the action of a voltage higher than a normal voltage, and the first memory may be, for example, an eprom memory, and the first memory may perform targeted storage on various characteristics of stored data, so that a storage space can be effectively saved, and a data characteristic table or a statistical table, etc. may be established according to actual needs, which is beneficial to data screening and data statistics. Specifically, after the parameters of the queried data are acquired, retrieval matching is performed in the first memory according to the parameters of the queried data, so that address information corresponding to the parameters of the queried data is obtained.
And 106, extracting data content corresponding to the parameters of the inquired data from a second memory according to the address information.
And step 108, outputting the data content.
In the embodiment of the invention, the second memory is a low-cost large-capacity memory device, such as a nandflash memory. Specifically, after address information corresponding to the parameter of the queried data is acquired, data content corresponding to the parameter of the queried data is extracted from the second memory according to the address information, and the data content is output.
In the embodiment of the invention, data is stored in a distributed manner in advance, so that when the data is queried, the parameter of the queried data is acquired, the parameter of the queried data is retrieved in the first memory according to the parameter of the queried data, the address information corresponding to the parameter of the queried data is acquired, then the data content corresponding to the parameter of the queried data is extracted from the second memory according to the address information and output, the data parameter is retrieved in the first memory and the corresponding address information is acquired, and the required data content is extracted from the second memory according to the address information, so that the data operation period can be shortened, and the timeliness is strong.
In some embodiments, the obtaining parameters of the queried data includes: and acquiring the type parameter of the data block and the characteristic parameter of the data block.
In the embodiment of the invention, the data in the second memory is stored by packing the data into data blocks. The type parameter of the data block may be, for example, an operation record, a bill record, internet data, and the like, and the characteristic parameter of the data block may be, for example, a user name, a bill number, or a storage date.
In some embodiments, as shown in fig. 2, the retrieving, in the first memory, the address information corresponding to the parameter of the queried data according to the parameter of the queried data includes:
Specifically, after the type parameter of the data block and the characteristic parameter of the data block are input into the first memory, the first memory performs information retrieval according to the received input type parameter of the data block and the characteristic parameter of the data block.
And step 204, if the characteristic parameters of the data block are not default.
And step 206, outputting address information of the data corresponding to the characteristic parameters of the data block.
In the embodiment of the invention, the default is that the system defaults to blank, and if not, the data block has unique characteristic parameters. Specifically, when searching in the first memory according to the type parameter of the data block and the characteristic parameter of the data block, if the characteristic parameter of the data block is not default, that is, the characteristic parameter of the data block is unique, the address information of the data corresponding to the characteristic parameter of the data block is output according to the characteristic parameter of the data block, since the characteristic parameter of the data block is unique, the output address information is also unique, and then the corresponding data content is extracted from the second memory according to the address information of the data corresponding to the characteristic parameter of the data block, and the data content is output. Therefore, the data operation period can be shortened, and the timeliness is strong.
In some other embodiments, as shown in fig. 3, the retrieving, in the first memory, the parameter of the queried data according to the parameter of the queried data to obtain the address information corresponding to the parameter of the queried data includes:
Specifically, after the type parameter of the data block and the characteristic parameter of the data block are input into the first memory, the first memory performs information retrieval according to the received input type parameter of the data block and the characteristic parameter of the data block.
Specifically, when retrieval is performed in the first memory according to the type parameter of the data block and the characteristic parameter of the data block, if the characteristic parameter of the data block is default, that is, the characteristic parameter of the data block is blank, address information of the batch data corresponding to the type parameter of the data block is output according to the type parameter of the data block, then all data contents corresponding to the type parameter of the data block are extracted from the second memory according to the address information of the batch data, and the data contents of the batch are output. Therefore, the data operation period can be shortened, and the timeliness is strong.
In some embodiments, as shown in fig. 4, the method further comprises:
In the embodiment of the invention, the data to be stored is structurally stored according to actual requirements, namely, a data structure with a specific format is established, and the data is packaged into data blocks according to different data structures and is stored in the second memory. Specifically, as shown in fig. 5, in the data block, the data to be stored is dynamically allocated according to the data stream required in real time, and the data to be stored may be stored in sequence, or the data to be stored may be randomly stored without performing continuous address space writing when the data is stored.
Specifically, the data to be stored is stored in the second memory in the form of data blocks according to a data structure with a specific format, and after the data blocks are stored in the second memory, the type parameters of the data blocks, the characteristic parameters of the data blocks, the address information of the data corresponding to the type parameters of the data blocks, and the address information of the data corresponding to the characteristic parameters of the data blocks are obtained.
In the embodiment of the present invention, the data link table area is used for storing parameter information and corresponding address information of different data blocks. Specifically, the type parameter of the data block, the characteristic parameter of the data block, and address information of data respectively corresponding to the type parameter of the data block and the characteristic parameter of the data block are stored in a data link table area.
In some embodiments, the method further comprises: acquiring a data structure of the data link table area; and storing the data link table area to a corresponding storage space in a first memory according to the data structure of the data link table area.
In the embodiment of the present invention, the data structure of the data link table area refers to the characteristic parameter of the data block and the address information of the data corresponding to the characteristic parameter of the data block. As shown in fig. 6, specifically, the data structure in the data link table region is obtained, the data link table region is stored into the corresponding storage space in the first storage according to the data structure, and the data parameter and the address information of the corresponding data are stored into the first storage in the form of the data link table region, which is beneficial to subsequent data screening and data statistics.
In some embodiments, the method further comprises: and pre-establishing a data link table area, wherein the data link table area is used for storing the type parameter of the data block, the characteristic parameter of the data block and address information of data corresponding to the type parameter of the data block and the characteristic parameter of the data block respectively.
Correspondingly, an embodiment of the present invention further provides a data processing apparatus based on distributed mode, as shown in fig. 7, where the apparatus 700 includes:
an obtaining module 702, configured to obtain parameters of the queried data;
a retrieving module 704, configured to retrieve, according to the parameter of the queried data, from a first memory, to obtain address information corresponding to the parameter of the queried data;
an extracting module 706, configured to extract data content corresponding to the parameter of the queried data from a second memory according to the address information;
an output module 708 configured to output the data content.
The distributed data processing device provided by the embodiment of the invention has the advantages that data are stored in a distributed manner in advance, so that when the data are inquired, the parameters of the inquired data are acquired through the acquisition module, the retrieval module retrieves the parameters of the inquired data in the first memory according to the parameters of the inquired data to acquire the address information corresponding to the parameters of the inquired data, the extraction module is used for extracting the data content corresponding to the parameters of the inquired data from the second memory according to the address information, the data content is output through the output module, the data parameters are retrieved from the first memory to acquire the corresponding address information, and the required data content is extracted from the second memory according to the address information, so that the data operation period can be shortened, and the timeliness is strong.
Optionally, in another embodiment of the apparatus, as shown in fig. 7, the apparatus 700 further includes:
and the storage module 710 is configured to pack data to be stored into a data block, and store the data block into the second memory.
And storing the type parameter of the data block, the characteristic parameter of the data block and address information of data corresponding to the type parameter of the data block and the characteristic parameter of the data block respectively into a data chain table area.
And storing the data link table area to a corresponding storage space in a first memory according to the data structure of the data link table area.
Optionally, in another embodiment of the apparatus, as shown in fig. 7, the apparatus 700 further includes:
the establishing module 712 is configured to establish a data link table area in advance, where the data link table area is used to store the type parameter of the data block, the characteristic parameter of the data block, and address information of data corresponding to the type parameter of the data block and the characteristic parameter of the data block, respectively.
Optionally, in other embodiments of the apparatus, the obtaining module 702 is specifically configured to:
and acquiring the type parameter of the data block and the characteristic parameter of the data block.
The method comprises the steps of obtaining type parameters of data blocks, characteristic parameters of the data blocks and address information of data corresponding to the type parameters of the data blocks and the characteristic parameters of the data blocks respectively.
And acquiring a data structure of the data link table area.
Optionally, in other embodiments of the apparatus, the retrieving module 704 is specifically configured to:
retrieving in a first memory according to the type parameter of the data block and the characteristic parameter of the data block;
if the characteristic parameters of the data block are not default;
outputting the address information of the data corresponding to the characteristic parameters of the data block.
Retrieving in a first memory according to the type parameter of the data block and the characteristic parameter of the data block;
if the characteristic parameters of the data block are default;
and outputting the address information of the batch data corresponding to the type parameter of the data block.
It should be noted that, the above-mentioned data processing apparatus based on distributed mode may execute the data processing method based on distributed mode provided by the embodiment of the present invention, and has corresponding functional modules and beneficial effects of the execution method, and reference may be made to the data processing method based on distributed mode provided by the embodiment of the present invention without detailed technical details described in the embodiment of the data processing apparatus based on distributed mode.
Fig. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention, and as shown in fig. 8, the electronic device 800 includes:
one or more processors 802 and a memory 804, one processor 802 being illustrated in fig. 8.
The processor 802 and the memory 804 may be connected by a bus or other means, such as by a bus in FIG. 8.
The memory 804, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the distributed data processing method according to the embodiments of the present invention (for example, the obtaining module 702, the retrieving module 704, the extracting module 706, the outputting module 708, the storing module 710, and the establishing module 712 shown in fig. 7). The processor 802 executes various functional applications and data processing of the electronic device, i.e., implements the distributed-based data processing method of the above-described method embodiments, by executing nonvolatile software programs, instructions, and modules stored in the memory 804.
The memory 804 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the distributed-based data processing apparatus use, and the like. Further, the memory 804 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 804 optionally includes memory located remotely from the processor 802, which may be connected to a distributed based data processing apparatus via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 804, and when executed by the one or more electronic devices 800, perform the distributed-based data processing method in any of the above-described method embodiments, for example, performing the above-described method steps 102 to 108 in fig. 1, method steps 202 to 206 in fig. 2, method steps 302 to 306 in fig. 3, and method steps 402 to 406 in fig. 4; the functionality of modules 702 to 712 in fig. 7 is implemented.
The electronic device in the embodiments of the present invention exists in various forms, including but not limited to:
(1) mobile communication devices, which are characterized by mobile communication capabilities and are primarily targeted at providing voice and data communications. Such terminals include smart phones (e.g., iphones), multimedia phones, functional phones, and low-end phones, among others.
(2) The ultra-mobile personal computer equipment belongs to the category of personal computers, has calculation and processing functions and generally has the characteristic of mobile internet access. Such terminals include PDA, MID, and UMPC devices, such as ipads.
(3) Portable entertainment devices such devices may display and play multimedia content. Such devices include audio and video players (e.g., ipods), handheld game consoles, electronic books, as well as smart toys and portable car navigation devices.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A distributed based data processing method, the method comprising:
acquiring parameters of queried data;
retrieving in a first memory according to the parameters of the queried data to obtain address information corresponding to the parameters of the queried data;
extracting data content corresponding to the parameters of the inquired data from a second memory according to the address information;
and outputting the data content.
2. The method of claim 1, wherein the obtaining parameters of the queried data comprises:
and acquiring the type parameter of the data block and the characteristic parameter of the data block.
3. The method according to claim 2, wherein the retrieving in the first memory according to the parameter of the queried data to obtain the address information corresponding to the parameter of the queried data includes:
retrieving in a first memory according to the type parameter of the data block and the characteristic parameter of the data block;
if the characteristic parameters of the data block are not default;
outputting the address information of the data corresponding to the characteristic parameters of the data block.
4. The method according to claim 2, wherein the retrieving in the first memory according to the parameter of the queried data to obtain the address information corresponding to the parameter of the queried data includes:
retrieving in a first memory according to the type parameter of the data block and the characteristic parameter of the data block;
if the characteristic parameters of the data block are default;
and outputting the address information of the batch data corresponding to the type parameter of the data block.
5. The method according to any one of claims 1-4, further comprising:
packing data to be stored into a data block, and storing the data block into the second memory;
acquiring type parameters of data blocks, characteristic parameters of the data blocks and address information of data corresponding to the type parameters of the data blocks and the characteristic parameters of the data blocks respectively;
and storing the type parameter of the data block, the characteristic parameter of the data block and address information of data corresponding to the type parameter of the data block and the characteristic parameter of the data block respectively into a data chain table area.
6. The method of claim 5, further comprising:
acquiring a data structure of the data link table area;
and storing the data link table area to a corresponding storage space in a first memory according to the data structure of the data link table area.
7. The method of claim 6, further comprising:
and pre-establishing a data link table area, wherein the data link table area is used for storing the type parameter of the data block, the characteristic parameter of the data block and address information of data corresponding to the type parameter of the data block and the characteristic parameter of the data block respectively.
8. A distributed based data processing apparatus, the apparatus comprising:
the acquisition module is used for acquiring parameters of the inquired data;
the retrieval module is used for retrieving in the first memory according to the parameters of the queried data to obtain address information corresponding to the parameters of the queried data;
the extraction module is used for extracting data content corresponding to the parameters of the inquired data from a second memory according to the address information;
and the output module is used for outputting the data content.
9. An electronic device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
10. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010237373.6A CN111444219A (en) | 2020-03-30 | 2020-03-30 | Distributed data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010237373.6A CN111444219A (en) | 2020-03-30 | 2020-03-30 | Distributed data processing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111444219A true CN111444219A (en) | 2020-07-24 |
Family
ID=71649256
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010237373.6A Pending CN111444219A (en) | 2020-03-30 | 2020-03-30 | Distributed data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111444219A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113031865A (en) * | 2021-03-23 | 2021-06-25 | 北京和利时系统工程有限公司 | Method and device for realizing data storage, computer storage medium and terminal |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030033308A1 (en) * | 2001-08-03 | 2003-02-13 | Patel Sujal M. | System and methods for providing a distributed file system utilizing metadata to track information about data stored throughout the system |
| CN101676899A (en) * | 2008-09-18 | 2010-03-24 | 上海宝信软件股份有限公司 | Profiling and inquiring method for massive database records |
| CN102375853A (en) * | 2010-08-24 | 2012-03-14 | 中国移动通信集团公司 | Distributed database system, method for building index therein and query method |
| CN105898207A (en) * | 2015-01-26 | 2016-08-24 | 杭州海康威视数字技术股份有限公司 | Intelligent processing method and system of video data |
| CN107704599A (en) * | 2017-10-16 | 2018-02-16 | 杭州安恒信息技术有限公司 | A kind of method and apparatus for realizing EDR data quick-searchings |
| CN110597762A (en) * | 2018-05-25 | 2019-12-20 | 杭州海康威视系统技术有限公司 | File processing method, device, equipment and storage medium |
-
2020
- 2020-03-30 CN CN202010237373.6A patent/CN111444219A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030033308A1 (en) * | 2001-08-03 | 2003-02-13 | Patel Sujal M. | System and methods for providing a distributed file system utilizing metadata to track information about data stored throughout the system |
| CN101676899A (en) * | 2008-09-18 | 2010-03-24 | 上海宝信软件股份有限公司 | Profiling and inquiring method for massive database records |
| CN102375853A (en) * | 2010-08-24 | 2012-03-14 | 中国移动通信集团公司 | Distributed database system, method for building index therein and query method |
| CN105898207A (en) * | 2015-01-26 | 2016-08-24 | 杭州海康威视数字技术股份有限公司 | Intelligent processing method and system of video data |
| CN107704599A (en) * | 2017-10-16 | 2018-02-16 | 杭州安恒信息技术有限公司 | A kind of method and apparatus for realizing EDR data quick-searchings |
| CN110597762A (en) * | 2018-05-25 | 2019-12-20 | 杭州海康威视系统技术有限公司 | File processing method, device, equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113031865A (en) * | 2021-03-23 | 2021-06-25 | 北京和利时系统工程有限公司 | Method and device for realizing data storage, computer storage medium and terminal |
| CN113031865B (en) * | 2021-03-23 | 2023-12-22 | 北京和利时系统集成有限公司 | Method, device, computer storage medium and terminal for realizing data storage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110399420B (en) | Export method, electronic equipment and medium of configurable Excel format | |
| CN105487760A (en) | Method and device for displaying message and electronic equipment | |
| CN104750620B (en) | A kind of internal memory migration method and device | |
| CN103139567A (en) | Image compressing and uncompressing method and device | |
| CN108388509A (en) | A kind of method for testing software, computer readable storage medium and terminal device | |
| CN112087487A (en) | Model training task scheduling method and device, electronic equipment and storage medium | |
| CN111443873A (en) | Method and device for managing Nand Flash memory | |
| CN103716237A (en) | Path-finding method and device utilizing binary heap sorting | |
| CN111444219A (en) | Distributed data processing method and device and electronic equipment | |
| CN103366013B (en) | A kind of method of data processing and server | |
| CN105740061B (en) | A kind of method, user equipment and system that GPU resource is rebuild | |
| CN105989114A (en) | Collection content recommendation method and terminal | |
| CN103997557A (en) | Address list management method and mobile terminal | |
| KR102623957B1 (en) | Method and device for providing item | |
| CN106682014B (en) | Game display data generation method and device | |
| CN107315658A (en) | A kind of data back up method and device, computer installation and readable storage medium storing program for executing | |
| CN105872731A (en) | Data processing method and device | |
| CN105282206A (en) | Method, device and system for processing website resource files | |
| CN106899558A (en) | The treating method and apparatus of access request | |
| CN105320609A (en) | Mobile terminal based data storage method and system | |
| CN108399266A (en) | Data pick-up method, apparatus, electronic equipment and computer readable storage medium | |
| CN106933934B (en) | Data table connection method and device | |
| CN108710514B (en) | Object jump control method and device, storage medium and electronic device | |
| WO2018177414A1 (en) | Data processing method, data processing device, computing device, and storage medium | |
| CN108668160A (en) | Rhythm barrage methods of exhibiting, device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200724 |