CN111782633B - Data processing method and device and electronic equipment - Google Patents
Data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN111782633B CN111782633B CN202010606946.8A CN202010606946A CN111782633B CN 111782633 B CN111782633 B CN 111782633B CN 202010606946 A CN202010606946 A CN 202010606946A CN 111782633 B CN111782633 B CN 111782633B
- Authority
- CN
- China
- Prior art keywords
- data
- fragments
- fragment
- initial
- slice
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 41
- 239000012634 fragment Substances 0.000 claims abstract description 322
- 238000004364 calculation method Methods 0.000 claims abstract description 40
- 238000012545 processing Methods 0.000 claims description 29
- 238000000034 method Methods 0.000 claims description 22
- 230000015654 memory Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 13
- 238000013135 deep learning Methods 0.000 abstract description 2
- 230000005012 migration Effects 0.000 description 23
- 238000013508 migration Methods 0.000 description 23
- 238000013467 fragmentation Methods 0.000 description 12
- 238000006062 fragmentation reaction Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 230000011218 segmentation Effects 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data processing method and device and electronic equipment, and relates to the technical fields of knowledge maps, deep learning and big data. The specific implementation scheme is as follows: when data retrieval is carried out, firstly, hash calculation is carried out on the data to be retrieved according to the number of current fragments, a first fragment identifier corresponding to the data to be retrieved is determined, the first fragment identifier is searched in a record, and then according to a search result, the first fragment corresponding to the first fragment identifier or a target fragment is obtained by carrying out hash calculation on the data to be retrieved according to the number of initial fragments to obtain the data to be retrieved on a second fragment corresponding to the second fragment identifier.
Description
Technical Field
The embodiment of the application relates to knowledge graph, deep learning and big data technology in the technical field of computers, in particular to a data processing method, a data processing device and electronic equipment.
Background
Distributed databases are typically created in the need for data volumes. In the case of a distributed database, when data is stored, it is common to store all data to be stored in pieces, and to collect the data when the data is retrieved.
As the amount of data to be stored increases, if the current shard is insufficient to store the data, the shard needs to be split, i.e. the data needs to be stored on more shards. When the data is searched, the data is searched in each fragment related to the data, the searching range is larger, and certain computing resources are occupied, so that the data searching efficiency is lower.
Disclosure of Invention
The application provides a data processing method, a data processing device and electronic equipment, which improve the data retrieval efficiency when data retrieval is carried out.
According to an aspect of the present application, there is provided a data processing method, which may include:
When a search instruction is received, carrying out hash calculation on data to be searched according to the number of current fragments, and determining a first fragment identifier corresponding to the data to be searched; the current fragments are obtained after splitting initial fragments, the number M of the current fragments is 2P times of the number N of the initial fragments, and N and P are integers greater than or equal to 1.
And searching whether the first fragment identifier corresponding to the data to be retrieved exists or not.
Determining the target fragment storing the data to be retrieved according to the searching result; the target fragments are first fragments or second fragments corresponding to the first fragment identifiers; and the second fragments are fragments corresponding to the second fragment identification by carrying out hash calculation on the data to be retrieved according to the number of the initial fragments.
And searching the data to be searched on the target fragment.
According to another aspect of the present application, there is provided a data processing apparatus, which may include:
the processing module is used for carrying out hash calculation on the data to be searched according to the number of current fragments when receiving a search instruction, and determining a first fragment identifier corresponding to the data to be searched; the current fragments are obtained after splitting initial fragments, the number M of the current fragments is 2P times of the number N of the initial fragments, and N and P are integers greater than or equal to 1.
The processing module is further used for searching whether the first fragment identifier corresponding to the data to be retrieved exists or not; determining the target fragment storing the data to be retrieved according to the searching result; the target fragments are first fragments or second fragments corresponding to the first fragment identifiers; and the second fragments are fragments corresponding to the second fragment identification by carrying out hash calculation on the data to be retrieved according to the number of the initial fragments.
And the retrieval module is used for retrieving the data to be retrieved on the target fragment.
According to another aspect of the present application, there is provided an electronic device, which may include:
at least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the data processing method of the first aspect described above.
According to another aspect of the present application, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the data processing method of the first aspect described above.
According to another aspect of the present application, there is provided a computer program product comprising: a computer program stored in a readable storage medium, from which it can be read by at least one processor of an electronic device, the at least one processor executing the computer program causing the electronic device to perform the data processing method of the first aspect described above.
According to the technical scheme, when data retrieval is carried out, hash calculation is carried out on the data to be retrieved according to the number of current fragments, a first fragment identifier corresponding to the data to be retrieved is determined, the first fragment identifier is searched in a record, and according to a search result, the first fragment corresponding to the first fragment identifier or a target fragment is obtained by carrying out hash calculation on the data to be retrieved according to the number of initial fragments to obtain the second fragment corresponding to the second fragment to retrieve the data to be retrieved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.
Drawings
The drawings are included to provide a better understanding of the present application and are not to be construed as limiting the application. Wherein:
fig. 1 is a flow chart schematically showing a data processing method according to a first embodiment of the present application;
FIG. 2 is a schematic illustration of a tile splitting provided in accordance with a first embodiment of the present application;
FIG. 3 is a flow chart of a data processing method according to a second embodiment of the present application;
FIG. 4 is a schematic illustration of a tile splitting provided in accordance with a second embodiment of the present application;
FIG. 5 is a schematic diagram of a tile splitting provided in accordance with a third embodiment of the present application;
Fig. 6 is a schematic diagram of a data processing apparatus according to a fourth embodiment of the present application;
Fig. 7 is a block diagram of an electronic device of a data processing method according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: there are three cases, a alone, a and B together, and B alone, wherein a, B may be singular or plural. In the text description of the present application, the character "/" generally indicates that the front-rear associated object is an or relationship.
The data processing method provided by the embodiment of the application can be applied to a scene of data retrieval. In the prior art, with the increase of the amount of data to be stored, if the current fragmentation is insufficient to store the data, fragmentation is required, that is, the data needs to be stored on more fragments. In this way, when data is retrieved, a search is performed on each fragment related to the data, and the search range is large, so that the data retrieval efficiency is low.
In order to improve the retrieval efficiency of data, the embodiment of the application provides a data processing method, when a retrieval instruction is received, hash calculation can be firstly carried out on the data to be retrieved according to the number of current fragments, and a first fragment identifier corresponding to the data to be retrieved is determined; the current fragments are obtained after splitting the initial fragments, and the number M of the current fragments is 2P times of the number N of the initial fragments; searching whether a first fragment identifier corresponding to the data to be searched exists or not; determining a target fragment storing the data to be retrieved according to the searching result; the target fragments are first fragments corresponding to the first fragment identifiers, or the target fragments are second fragments corresponding to the second fragment identifiers obtained by carrying out hash calculation on the data to be retrieved according to the number of the initial fragments; and then searching the data to be searched on the target fragment. Wherein N and P are integers greater than or equal to 1.
Therefore, when the data processing method provided by the embodiment of the application is used for searching the data, firstly, hash calculation is carried out on the data to be searched according to the number of current fragments, a first fragment identifier corresponding to the data to be searched is determined, the first fragment identifier is searched in a record, and then, according to a search result, the first fragment corresponding to the first fragment identifier or a target fragment is used for carrying out hash calculation on the data to be searched according to the number of initial fragments to obtain the data to be searched on the second fragment corresponding to the second fragment, and compared with the search in each fragment related to the data in the prior art, the search range is smaller, and although the hash calculation is possibly carried out on the data to be searched for twice, the calculation resources occupied by the hash calculation for the two times are less, so that the data search efficiency is improved to a certain extent.
The data processing method provided by the application will be described in detail by specific examples. It is to be understood that the following embodiments may be combined with each other and that some embodiments may not be repeated for the same or similar concepts or processes.
Example 1
Fig. 1 is a flow chart illustrating a data processing method according to a first embodiment of the present application, which can be applied to the technical field of big data. The data processing method may be performed by software and/or hardware means, which may be provided in the data processing means, for example. The data processing apparatus may be an electronic device, for example. Referring to fig. 1, the data processing method may include:
S101, when a search instruction is received, carrying out hash calculation on data to be searched according to the number of current fragments, and determining a first fragment identifier corresponding to the data to be searched.
The current fragments are obtained after the initial fragments are split, the number M of the current fragments is 2P times of the number N of the initial fragments, and N and P are integers greater than or equal to 1.
For example, the search instruction may be input by a user, that is, search of the data to be searched is performed after receiving a search operation input by the user.
In the embodiment of the present application, when the slices are split, the number N of the initial slices is multiplied by 2P to obtain the number M of the current slices, which has the following advantages: each tile can be split separately before and after each change. Assuming that the initial slice includes slice 0 and slice 1, after the slice is split, the number of current slices is 4, then the data appearing on slice 0 and slice 2 appears on slice 0 before the data appearing on slice 1 and slice 3 appears on slice 1 before the data appearing on slice 1; in other words, slice 0 may be split into slice 0 and slice 2 separately, slice 1 may be split into slice 1 and slice 3 separately, and the splitting of each slice is performed separately and does not affect each other, so, compared with the prior art, in the embodiment of the present application, it is not necessary to recalculate all data on the slice before splitting, thereby improving the efficiency of the splitting of the slice to some extent.
Taking the initial number of slices as 4, and taking the 4 slices as a slice 1, a slice 2, a slice 3 and a slice 4 as examples, the slice 1 stores data A and data B, the slice 2 stores data C and data D, the slice 3 stores data E and data F, and the slice 4 stores data G and data H. When a certain slice in the four slices is supposed to split the slice 1, the number of the initial slices is multiplied by 2P times to obtain the number of the current slices after splitting, and partial data on the slice 1 is migrated to the newly added slice. Assuming that P is equal to 1, i.e. the number of initial slices is 4 times 2, the number of current slices is 8, and data B on slice 1 is migrated to the newly added slice 5, and this slice 5 is recorded. For example, referring to fig. 2, fig. 2 is a schematic diagram of a slice splitting provided in accordance with a first embodiment of the present application, it can be seen that, compared to an initial slice, 4 new slices are added to a current slice after the slice splitting, and a specific slice splitting process and a data migration process thereof will be described in detail in a second embodiment described below.
Referring to fig. 2, after the fragments 1 are split to obtain 8 fragments, if the data to be retrieved is the data B, when retrieving the data B, hash calculation may be performed on the data B according to the current number of fragments 8 to obtain the fragment identifier 5 corresponding to the data B, and then find whether the fragment identifier 5 corresponding to the data B exists in the record.
S102, searching whether a first fragment identifier corresponding to the data to be retrieved exists or not.
The first slice identifier may be a serial number of the first slice, or may be a serial number of the first slice, so long as the first slice can be distinguished, and herein, the specific mode of the first slice identifier is described, which is not limited in the embodiments of the present application.
S103, determining the target fragment storing the data to be retrieved according to the search result.
The target fragments are first fragments or second fragments corresponding to the first fragment identifiers; and the second fragments are fragments corresponding to the second fragment identification obtained by carrying out hash calculation on the data to be retrieved according to the number of the initial fragments.
For example, when determining the target fragment storing the data to be retrieved according to the search result, if the first fragment identifier is found, it is indicated that the data to be retrieved is stored on the first fragment, and data migration occurs before the data to be retrieved, where the first fragment may be directly determined as the target fragment; if the first fragment identifier is not found, it indicates that the data to be retrieved cannot be stored on the first fragment, and data migration does not occur before the data to be retrieved, so that hash calculation is needed to be performed on the data to be retrieved according to the number N of the initial fragments to obtain a second fragment identifier, and the second fragment corresponding to the second fragment identifier is determined to be the target fragment.
With continued reference to fig. 2, when searching whether the slice identifier 5 corresponding to the data B exists in the record, since the slice identifier 5 corresponding to the slice 5 is already recorded in the record while the data B is migrated to the slice 5, the slice identifier 5 can be searched in the record, the slice 5 is determined as a target slice, and then the data B is searched on the slice 5.
Continuing to combine with the illustration of fig. 2, assuming that the data to be retrieved is data E, when retrieving the data E, the hash calculation can be performed on the data E according to the current number of fragments 8 to obtain the fragment identifier 7 corresponding to the data E, and then searching whether the fragment identifier 7 corresponding to the data E exists in the record. When searching whether the fragment identifier 7 corresponding to the data E exists in the record, as the fragment 3 where the data E is located is not split and the data E is not migrated, the fragment identifier 7 cannot be recorded in the record, in this case, hash calculation is performed on the data E according to the number 4 of initial fragments, the fragment identifier 3 corresponding to the data E can be obtained, the fragment 3 corresponding to the fragment identifier 3 is determined as a target fragment, and then the data E is searched on the fragment 3.
After determining the target fragment storing the data to be searched according to the search result, the data to be searched may be searched on the determined target fragment, that is, the following S104 is performed, thereby completing the search of the data to be searched.
S104, searching the data to be searched on the target fragment.
Therefore, when data retrieval is performed, firstly, hash calculation is performed on the data to be retrieved according to the number of current fragments, a first fragment identifier corresponding to the data to be retrieved is determined, the first fragment identifier is searched in a record, and then, according to a search result, the first fragment corresponding to the first fragment identifier or a target fragment is obtained by performing hash calculation on the data to be retrieved according to the number of initial fragments to obtain the data to be retrieved on a second fragment corresponding to the second fragment identifier.
Based on the embodiment shown in fig. 1, before hash calculation is performed on the data to be retrieved according to the number of current slices obtained by the slice splitting, the current slices can be obtained by the slice splitting only by performing the slice splitting and data migration on a certain slice in the initial slices, and the number of current slices is determined. In the prior art, when the fragmentation is split, as all data on the fragments before the splitting can be recalculated, the data of the fragmentation change needs to be further subjected to data migration, so that the fragmentation splitting efficiency is lower, and the data migration efficiency is lower.
In order to improve the fragmentation efficiency and the data migration efficiency, and thus improve the data retrieval efficiency, the embodiment of the application also provides a data processing method, which does not need to recalculate all data on the fragments before fragmentation compared with the prior art, so that the fragmentation efficiency can be improved to a certain extent; and when data migration is performed, the data volume to be migrated is small, so that the data migration efficiency can be improved to a certain extent, and therefore, the data retrieval efficiency can be improved to a certain extent.
Example two
Fig. 3 is a flowchart of a data processing method according to a second embodiment of the present application, which may also be executed by software and/or hardware devices, and as an example, referring to fig. 3, the data processing method may include:
S301, determining the number of initial fragments according to the data quantity of the initial data.
Wherein the initial data comprises data to be retrieved.
For example, in determining the number of initial tiles from the data amount of the initial data, a power of 2 may be selected as the number of initial tiles. For example, the number of initial fragments may be 2, 4, or 8, or of course, 16, or 32, or the like, and specifically, the number of initial fragments may be determined reasonably according to the data amount of the initial data.
Continuing with the illustration of fig. 2, assume that the initial data includes data a, data B, data C, data D, data E, data F, data G, and data H, the number of initial slices determined from the data amount of the initial data is 4, and the 4 slices are slice 1, slice 2, slice 3, and slice 4, respectively.
After determining the initial data and the number of the corresponding initial fragments respectively, the hash calculation may be performed on the initial data according to the number of the initial fragments, and the initial data may be uniformly stored on different fragments in the initial fragments, that is, the following S302 is performed:
S302, carrying out hash calculation on the initial data, and uniformly storing the initial data on different fragments in the initial fragments.
Continuing with the description in S301, hash computation may be performed on the data a, the data B, the data C, the data D, the data E, the data F, the data G, and the data H according to the number 4 of initial fragments, to determine the fragment identifier corresponding to each data, and store each data on the fragment corresponding to the fragment identifier. After hash calculation, the fragment identifiers corresponding to the data A and the data B are assumed to be fragment identifiers 1, the data A and the data B are stored on the fragment 1, the fragment identifiers corresponding to the data C and the data D are assumed to be fragment identifiers 2, the data C and the data D are stored on the fragment 2, the fragment identifiers corresponding to the data E and the data F are assumed to be fragment identifiers 3, the data E and the data F are stored on the fragment 3, the fragment identifiers corresponding to the data G and the data H are assumed to be fragment identifiers 4, and the data G and the data H are stored on the fragment 4, so that the fragment storage of initial data is completed.
S303, when the third slicing in the initial slicing meets the preset condition and the third slicing is split, multiplying the number N of the initial slicing by 2P to obtain the number M of the current slicing.
Wherein the current slice includes an initial slice and a newly added slice, and the splitting of the third slice is the first splitting of the initial slice. For example, the preset conditions include: the amount of data is greater than a first threshold and/or the amount of data of the hotspot data included in the data is greater than a second threshold.
When determining whether a certain slice in the initial slices is needed, if a third slice is supposed to split, judging whether the data amount on the third slice is larger than a first threshold value, if the data amount on the third slice is larger than the first threshold value, indicating that the data amount on the third slice is larger, and if the data amount on the third slice is larger than the first threshold value, splitting the third slice is needed; and/or judging whether the data quantity of the hot spot data on the third slice is larger than a second threshold value, if the data quantity of the hot spot data on the third slice is larger than the second threshold value, indicating that the hot spot data on the third slice is more, and if the hot spot data needs to be scattered, the third slice needs to be split. It may be appreciated that in the embodiment of the present application, when determining whether the third slice needs to be split, the method includes, with preset conditions: the description is made by taking the case that the data amount is greater than the first threshold value and/or the data amount of the hot spot data included in the data is greater than the second threshold value as an example, and the setting can be specifically performed according to actual needs.
When it is determined that the third slice needs to be split, the number N of the initial slices may be multiplied by 2P to obtain the number M of the current slices, thereby obtaining the number of the current slices. In the embodiment of the present application, when the slices are split, the number N of the initial slices is multiplied by 2P to obtain the number M of the current slices, which has the following advantages: each tile can be split separately before and after each change. Assuming that the initial slice includes slice 0 and slice 1, after the slice is split, the number of current slices is 4, then the data appearing on slice 0 and slice 2 appears on slice 0 before the data appearing on slice 1 and slice 3 appears on slice 1 before the data appearing on slice 1; in other words, slice 0 may be split into slice 0 and slice 2 separately, slice 1 may be split into slice 1 and slice 3 separately, and the splitting of each slice is performed separately and does not affect each other, so, compared with the prior art, in the embodiment of the present application, it is not necessary to recalculate all data on the slice before splitting, thereby improving the efficiency of the splitting of the slice to some extent.
Continuing with the description in S301 above, assuming that the slice 1 is split, the number of initial slices is multiplied by 2P times to obtain the number of current slices after splitting, and the partial data on the slice 1 is migrated to a newly added slice. Assuming that P is equal to 1, that is, the number of initial slices is 4 times 2, the number of current slices is 8, and data B on slice 1 is migrated to a slice newly added, for example, please refer to fig. 4, fig. 4 is a schematic diagram of slice splitting provided according to a second embodiment of the present application, it can be seen that, compared with the initial slices, the current slices after slice splitting are added with 4 new slices, and data B on slice 1 can be migrated to a slice in the newly added 4 slices.
It may be understood that, in the embodiment of the present application, when migrating the data B on the slice 1 to any one of the newly added 4 slices, instead of migrating the data B to any one of the newly added 4 slices, it is necessary to determine, according to the number N of the initial slices, the slice corresponding to the slice 1 in the newly added slices, and migrate the data B to the slice corresponding to the slice 1 in the newly added slices, that is, execute the following S304:
S304, determining a first fragment corresponding to the third fragment in the newly added fragments according to the number N of the initial fragments, migrating the data to be searched on the third fragment to the first fragment, and recording the first fragment identification.
Continuing with the description in fig. 4, before the migration of the data B, determining the corresponding slice of the slices 1 in the newly added 4 slices according to the number 4 of the initial slices is needed, for example, a remainder method may be adopted to determine that the corresponding slice of the slices 1 in the newly added 4 slices is the slice 5, and after the slice 5 is determined, the data B is migrated to the slice 5, which can be seen in fig. 2, so as to complete the slice splitting of the slices 1 and the corresponding data migration, where the slice splitting of the slices 1 may be marked as the first splitting of the initial slices.
Therefore, in the embodiment of the application, when the splitting of the fragments is carried out, as the splitting of each fragment is carried out independently and does not affect each other, compared with the prior art, all data on the fragments before the splitting are not required to be recalculated, and the splitting efficiency of the fragments can be improved to a certain extent; when data migration is carried out, only partial data on the fragments with the fragmentation is required to be subjected to data migration, other data are not required to be subjected to data migration, the data migration efficiency can be improved to a certain extent due to the fact that the data quantity to be migrated is small, in addition, as the fragmentation and the data migration are carried out in the data retrieval process, the influence of the fragmentation and the data migration on the data retrieval process can be reduced along with the improvement of the fragmentation and the data migration efficiency, and therefore the data retrieval efficiency can be improved to a certain extent.
Based on the embodiment shown in fig. 3, for the initial slicing, when the initial slicing splitting is performed, the number of the initial slicing may be multiplied by 2P times to obtain the number of current slicing after splitting, so as to complete the initial slicing splitting of the initial slicing. After the first slice splitting is completed, if a certain slice in the initial slice is split, and it is assumed that the fourth slice is split again by the first slice, where Q is greater than or equal to 2 and less than or equal to N, then the number of the initial slices is not required to be multiplied by 2P times, and instead, the first slice is split directly by using the slice with increased first slice, and the data migration is performed.
Example III
When the fourth fragment in the initial fragments meets a preset condition and fragments the fourth fragment, determining a fifth fragment corresponding to the fourth fragment in the newly added fragments according to the number N of the initial fragments; splitting of the fourth slice is the Q-th splitting of the initial slice; and migrating part of data on the fourth slice to the fifth slice, and recording a fifth slice identifier corresponding to the fifth slice.
When determining whether a certain slice in the initial slices is needed, if a fourth slice is supposed to split, judging whether the data amount on the fourth slice is larger than a first threshold value, if the data amount on the fourth slice is larger than the first threshold value, indicating that the data amount on the fourth slice is larger, and if the data amount on the fourth slice is larger than the first threshold value, splitting the fourth slice; and/or judging whether the data quantity of the hot spot data on the fourth slice is larger than a second threshold value, if the data quantity of the hot spot data on the fourth slice is larger than the second threshold value, indicating that the hot spot data on the fourth slice is more, and if the hot spot data needs to be scattered, the fourth slice needs to be split. It may be appreciated that in the embodiment of the present application, when determining whether the fourth slice needs to be split, the method includes, with preset conditions: the description is made by taking the case that the data amount is greater than the first threshold value and/or the data amount of the hot spot data included in the data is greater than the second threshold value as an example, and the setting can be specifically performed according to actual needs.
Therefore, in the embodiment of the application, when the fourth slice is split, the number of initial slices is not needed, or the current slices are multiplied by 2P times to split the slices, but the fourth slice is split into the fourth slice and the fifth slice directly, and part of data on the fourth slice is migrated to the fifth slice, namely the slices added by the first split are directly used for splitting the slices and data migration, so that the efficiency of the slice splitting and the data migration can be improved to a certain extent.
Continuing with the description in S301 above, after the initial slicing of the slice 1 is completed, assuming that the slice 2 needs to be split again, unlike the slice 1 described above, the number of the initial slices is not required to be split again, or the number of the current slices is multiplied by 2P times to determine that the corresponding slice of the slice 2 is the slice 6 in the newly added 4 slices directly by adopting the remainder method, and after determining the slice 6, migrating the data C to the slice 6, for example, as can be seen in fig. 5, fig. 5 is a schematic diagram of the slicing provided by the third embodiment of the present application, it can be seen that compared with the slice after the first slicing, the current slice after the slicing is not added with a new slice, but only the slice 2 is split into the slice 2 and the slice 3, and migrating the data C on the slice 2 to the slice 6; and recording the segmentation mark 6 of the segmentation 6, namely, directly using the segmentation added by the first segmentation to carry out segmentation and data migration, so that the segmentation and data migration efficiency can be improved to a certain extent.
It can be understood that by recording the slice identifier 6 corresponding to the slice 6, the purpose is that: when the data C is searched later, if the fragment identifier 6 corresponding to the data C exists in the record, if the fragment identifier 6 is searched in the record, the fragment 6 is determined to be the target fragment, and then the data C is searched on the fragment 6.
Example IV
Fig. 6 is a schematic structural diagram of a data processing apparatus 60 according to a fourth embodiment of the present application, and as shown in fig. 6, the data processing apparatus 60 may include:
The processing module 601 is configured to perform hash computation on data to be retrieved according to the number of current fragments when a retrieval instruction is received, and determine a first fragment identifier corresponding to the data to be retrieved; the current fragments are obtained after the initial fragments are split, the number M of the current fragments is 2P times of the number N of the initial fragments, and N and P are integers greater than or equal to 1.
The processing module 601 is further configured to find whether a first fragment identifier corresponding to the data to be retrieved exists; determining a target fragment storing the data to be retrieved according to the searching result; the target fragments are first fragments or second fragments corresponding to the first fragment identifiers; and the second fragments are fragments corresponding to the second fragment identification obtained by carrying out hash calculation on the data to be retrieved according to the number of the initial fragments.
The retrieving module 602 is configured to retrieve data to be retrieved on the target tile.
Optionally, the processing module 601 is specifically configured to determine the first slice as the target slice if the first slice identifier is found; if the first fragment identification is not found, carrying out hash calculation on the data to be retrieved according to the number N of the initial fragments to obtain a second fragment identification, and determining the second fragment corresponding to the second fragment identification as a target fragment.
Optionally, the processing module 601 is further configured to multiply the number N of the initial fragments by 2P to obtain the number M of the current fragments when the third fragment in the initial fragments meets a preset condition and fragments the third fragment; the current slicing comprises initial slicing and newly added slicing; the splitting of the third slice is the first splitting of the initial slice.
The processing module 601 is further configured to determine, according to the number N of initial slices, a first slice corresponding to the third slice in the newly added slices; and migrating the data to be retrieved on the third fragment to the first fragment, and recording the first fragment identification.
Optionally, the processing module 601 is further configured to determine, when the fourth slice in the initial slices meets a preset condition and the fourth slice is split, a fifth slice corresponding to the fourth slice in the newly added slices according to the number N of the initial slices; migrating part of data on the fourth segment to a fifth segment, and recording a fifth segment identifier corresponding to the fifth segment; the splitting of the fourth slice is the Q-th splitting of the initial slice, Q is greater than or equal to 2 and less than or equal to N.
Optionally, the preset conditions include: the amount of data is greater than a first threshold and/or the amount of data of the hotspot data included in the data is greater than a second threshold.
Optionally, the processing module 601 is further configured to determine the number of initial fragments according to the data amount of the initial data; wherein the initial data comprises data to be retrieved; and hash calculation is carried out on the initial data, and the initial data is uniformly stored on different fragments in the initial fragments.
The data processing device 60 provided in the embodiment of the present application may execute the technical scheme of the data processing method in any of the above embodiments, and the implementation principle and beneficial effects of the data processing method are similar to those of the data processing method, and reference may be made to the implementation principle and beneficial effects of the data processing method, and no further description is given here.
The present application also provides a computer program product comprising: the computer program is stored in the readable storage medium, and the at least one processor of the electronic device may read the computer program from the readable storage medium, where execution of the computer program by the at least one processor causes the electronic device to execute the scheme provided in any of the foregoing embodiments, and the implementation principle and the beneficial effects of the data processing method are similar to those of the implementation principle and the beneficial effects of the data processing method, which are not described herein again.
According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.
As shown in fig. 7, fig. 7 is a block diagram of an electronic device of a data processing method according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.
As shown in fig. 7, the electronic device includes: one or more processors 701, memory 702, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 701 is illustrated in fig. 7.
Memory 702 is a non-transitory computer readable storage medium provided by the present application. Wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the data processing method provided by the present application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to execute the data processing method provided by the present application.
The memory 702 is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules (e.g., the processing module 601 and the retrieval module 602 shown in fig. 6) corresponding to the data processing method according to the embodiment of the present application. The processor 701 executes various functional applications of the server and data processing, i.e., implements the data processing method in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 702.
Memory 702 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the electronic device of the data processing method, and the like. In addition, the memory 702 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 702 optionally includes memory remotely located relative to processor 701, which may be connected to the data processing method's electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the data processing method may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected by a bus or otherwise, in fig. 7 by way of example.
The input device 703 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device of the data processing method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, etc. The output device 704 may include a display apparatus, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, when data retrieval is carried out, firstly, hash calculation is carried out on the data to be retrieved according to the number of current fragments, a first fragment identifier corresponding to the data to be retrieved is determined, the first fragment identifier is searched in a record, and then according to a searching result, the first fragment corresponding to the first fragment identifier or a target fragment is obtained by carrying out hash calculation on the data to be retrieved according to the number of initial fragments to obtain the data to be retrieved in a second fragment corresponding to the second fragment identifier.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed embodiments are achieved, and are not limited herein.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.
Claims (15)
1. A data processing method, comprising:
When a search instruction is received, carrying out hash calculation on data to be searched according to the number of current fragments, and determining a first fragment identifier corresponding to the data to be searched; the current fragments are obtained after splitting initial fragments, the number M of the current fragments is 2P times of the number N of the initial fragments, and N and P are integers greater than or equal to 1;
Searching whether the first fragment identifier corresponding to the data to be retrieved exists or not;
Determining the target fragment storing the data to be retrieved according to the searching result; the target fragments are first fragments or second fragments corresponding to the first fragment identifiers; the second fragments are fragments corresponding to second fragment identifiers obtained by carrying out hash calculation on the data to be retrieved according to the number of the initial fragments;
and searching the data to be searched on the target fragment.
2. The method of claim 1, wherein the determining, according to the search result, the target tile storing the data to be retrieved includes:
if the first fragment identification is found, determining the first fragment as the target fragment;
if the first fragment identifier is not found, carrying out hash calculation on the data to be retrieved according to the number N of the initial fragments to obtain the second fragment identifier, and determining the second fragment corresponding to the second fragment identifier as the target fragment.
3. The method of claim 1, the method further comprising:
When a third fragment in the initial fragments meets a preset condition and fragments the third fragment, multiplying the number N of the initial fragments by 2P to obtain the number M of the current fragments; the current fragments comprise the initial fragments and newly added fragments; splitting of the third segment into a first split of the initial segment;
Determining the first fragments corresponding to the third fragments in the newly added fragments according to the number N of the initial fragments;
and migrating the data to be retrieved on the third fragment to the first fragment, and recording the first fragment identification.
4. A method according to claim 3, the method further comprising:
When a fourth fragment in the initial fragments meets a preset condition and fragments the fourth fragment, determining a fifth fragment corresponding to the fourth fragment in the newly added fragments according to the number N of the initial fragments; splitting of the fourth slice is the Q-th splitting of the initial slice, wherein Q is greater than or equal to 2 and less than or equal to N;
And migrating part of data on the fourth slice to the fifth slice, and recording a fifth slice identifier corresponding to the fifth slice.
5. A method according to claim 3,
The preset conditions include: the amount of data is greater than a first threshold and/or the amount of data of the hotspot data included in the data is greater than a second threshold.
6. The method of any one of claims 1-5, further comprising:
Determining the number of the initial fragments according to the data quantity of the initial data; wherein the initial data comprises the data to be retrieved;
And carrying out hash calculation on the initial data, and uniformly storing the initial data on different fragments in the initial fragments.
7. A data processing apparatus comprising:
The processing module is used for carrying out hash calculation on the data to be searched according to the number of current fragments when receiving a search instruction, and determining a first fragment identifier corresponding to the data to be searched; the current fragments are obtained after splitting initial fragments, the number M of the current fragments is 2P times of the number N of the initial fragments, and N and P are integers greater than or equal to 1;
The processing module is further used for searching whether the first fragment identifier corresponding to the data to be retrieved exists or not; determining the target fragment storing the data to be retrieved according to the searching result; the target fragments are first fragments or second fragments corresponding to the first fragment identifiers; the second fragments are fragments corresponding to second fragment identifiers obtained by carrying out hash calculation on the data to be retrieved according to the number of the initial fragments;
And the retrieval module is used for retrieving the data to be retrieved on the target fragment.
8. The apparatus of claim 7, wherein,
The processing module is specifically configured to determine the first slice as the target slice if the first slice identifier is found; if the first fragment identifier is not found, carrying out hash calculation on the data to be retrieved according to the number N of the initial fragments to obtain the second fragment identifier, and determining the second fragment corresponding to the second fragment identifier as the target fragment.
9. The device according to claim 7,
The processing module is further configured to multiply the number N of the initial fragments by 2P to obtain the number M of the current fragments when a third fragment in the initial fragments meets a preset condition and fragments the third fragment; the current fragments comprise the initial fragments and newly added fragments; splitting of the third segment into a first split of the initial segment;
The processing module is further configured to determine, according to the number N of the initial slices, the first slice corresponding to the third slice in the newly added slices; and migrating the data to be retrieved on the third segment to the first segment, and recording the first segment identification.
10. An apparatus according to claim 9,
The processing module is further configured to determine, when a fourth slice in the initial slices meets a preset condition and the fourth slice is split, a fifth slice corresponding to the fourth slice in the newly added slices according to the number N of the initial slices; migrating part of data on the fourth segment to the fifth segment, and recording a fifth segment identifier corresponding to the fifth segment; the splitting of the fourth slice is the Q-th splitting of the initial slice, Q is greater than or equal to 2 and less than or equal to N.
11. An apparatus according to claim 9,
The preset conditions include: the amount of data is greater than a first threshold and/or the amount of data of the hotspot data included in the data is greater than a second threshold.
12. The device according to any one of claim 7 to 11,
The processing module is further used for determining the number of the initial fragments according to the data quantity of the initial data; wherein the initial data comprises the data to be retrieved; and performing hash calculation on the initial data, and uniformly storing the initial data on different fragments in the initial fragments.
13. An electronic device, comprising:
at least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the data processing method of any one of claims 1-6.
14. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the data processing method of any one of claims 1-6.
15. A computer program product comprising a computer program which, when executed by a processor, performs the data processing method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010606946.8A CN111782633B (en) | 2020-06-29 | 2020-06-29 | Data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010606946.8A CN111782633B (en) | 2020-06-29 | 2020-06-29 | Data processing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111782633A CN111782633A (en) | 2020-10-16 |
| CN111782633B true CN111782633B (en) | 2024-04-30 |
Family
ID=72760853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010606946.8A Active CN111782633B (en) | 2020-06-29 | 2020-06-29 | Data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111782633B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112698926B (en) * | 2021-03-25 | 2021-07-02 | 成都新希望金融信息有限公司 | Data processing method, device, equipment, storage medium and system |
| CN117478699B (en) * | 2023-10-31 | 2024-07-12 | 北京中元易尚科技有限公司 | Data writing method based on multi-network fusion Internet of things card and computer equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2583069A1 (en) * | 2006-06-30 | 2007-12-30 | Avaya Technology Llc | Internet protocol telephony architecture including information storage and retrieval system to track fluency |
| CN101964041A (en) * | 2010-09-25 | 2011-02-02 | 合肥工业大学 | Perceptual hashing-based practical and safe image forensic system and forensic method |
| CN104836816A (en) * | 2015-06-04 | 2015-08-12 | 成都果豆数字娱乐有限公司 | Mobile equipment data storage method |
| WO2016197994A1 (en) * | 2015-12-03 | 2016-12-15 | 中兴通讯股份有限公司 | Capacity expansion method and device |
| CN106886599A (en) * | 2017-02-28 | 2017-06-23 | 北京京东尚科信息技术有限公司 | Image search method and device |
| CA3077830A1 (en) * | 2016-12-05 | 2018-06-05 | Avigilon Coporation | System and method for appearance search |
| WO2018099397A1 (en) * | 2016-12-01 | 2018-06-07 | 腾讯科技（深圳）有限公司 | Method and device for data migration in database cluster and storage medium |
| CN109952753A (en) * | 2016-10-25 | 2019-06-28 | 苹果公司 | For managing the user interface of the access to the voucher used in operation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AUPR464601A0 (en) * | 2001-04-30 | 2001-05-24 | Commonwealth Of Australia, The | Shapes vector |
-
2020
- 2020-06-29 CN CN202010606946.8A patent/CN111782633B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2583069A1 (en) * | 2006-06-30 | 2007-12-30 | Avaya Technology Llc | Internet protocol telephony architecture including information storage and retrieval system to track fluency |
| CN101964041A (en) * | 2010-09-25 | 2011-02-02 | 合肥工业大学 | Perceptual hashing-based practical and safe image forensic system and forensic method |
| CN104836816A (en) * | 2015-06-04 | 2015-08-12 | 成都果豆数字娱乐有限公司 | Mobile equipment data storage method |
| WO2016197994A1 (en) * | 2015-12-03 | 2016-12-15 | 中兴通讯股份有限公司 | Capacity expansion method and device |
| CN109952753A (en) * | 2016-10-25 | 2019-06-28 | 苹果公司 | For managing the user interface of the access to the voucher used in operation |
| WO2018099397A1 (en) * | 2016-12-01 | 2018-06-07 | 腾讯科技（深圳）有限公司 | Method and device for data migration in database cluster and storage medium |
| CA3077830A1 (en) * | 2016-12-05 | 2018-06-05 | Avigilon Coporation | System and method for appearance search |
| CN106886599A (en) * | 2017-02-28 | 2017-06-23 | 北京京东尚科信息技术有限公司 | Image search method and device |
Non-Patent Citations (2)
| Title |
|---|
| 多维数据近似检索的分层LSH索引算法模型研究;房华蓉;;电脑知识与技术(02);全文 * |
| 大数据相似性连接查询技术研究进展;马友忠;张智辉;林春杰;;计算机应用(04);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111782633A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111522967B (en) | Knowledge graph construction method, device, equipment and storage medium | |
| CN111177476B (en) | Data query method, device, electronic equipment and readable storage medium | |
| CN112559870B (en) | Multi-model fusion method, device, electronic equipment and storage medium | |
| CN111783952A (en) | Configuration method, device, system, electronic equipment and storage medium | |
| CN111782633B (en) | Data processing method and device and electronic equipment | |
| CN111158666B (en) | Entity normalization processing method, device, equipment and storage medium | |
| CN111523007A (en) | User interest information determination method, device, equipment and storage medium | |
| CN111340219A (en) | Neural network model searching method and device, image processing method and processor | |
| JP7241122B2 (en) | Smart response method and device, electronic device, storage medium and computer program | |
| CN110765075A (en) | Storage method and equipment of automatic driving data | |
| CN112115313B (en) | Regular expression generation and data extraction methods, devices, equipment and media | |
| CN111752960B (en) | Data processing method and device | |
| CN111767442B (en) | Data updating method, device, search server, terminal and storage medium | |
| CN111259058B (en) | Data mining method, data mining device and electronic equipment | |
| CN112446728B (en) | Advertisement recall method, device, equipment and storage medium | |
| CN111177479A (en) | Method and device for acquiring feature vectors of nodes in relational network graph | |
| CN112364185A (en) | Method and device for determining characteristics of multimedia resource, electronic equipment and storage medium | |
| CN112541934A (en) | Image processing method and device | |
| CN111523000B (en) | Method, apparatus, device and storage medium for importing data | |
| CN111340222B (en) | Neural network model searching method and device and electronic equipment | |
| CN112270412B (en) | Network operator processing method and device, electronic equipment and storage medium | |
| CN112507100B (en) | Update processing method and device of question-answering system | |
| CN112148279B (en) | Log information processing method, device, electronic equipment and storage medium | |
| CN110515964A (en) | A kind of file updating method, device, electronic equipment and medium | |
| CN112328807A (en) | Anti-cheating method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |