CN110019199A - Data storage, querying method, device, equipment, computer readable storage medium - Google Patents
Data storage, querying method, device, equipment, computer readable storage medium Download PDFInfo
- Publication number
- CN110019199A CN110019199A CN201710911189.3A CN201710911189A CN110019199A CN 110019199 A CN110019199 A CN 110019199A CN 201710911189 A CN201710911189 A CN 201710911189A CN 110019199 A CN110019199 A CN 110019199A
- Authority
- CN
- China
- Prior art keywords
- rowkey
- region
- user
- data
- user data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000013500 data storage Methods 0.000 title claims abstract description 12
- 230000002123 temporal effect Effects 0.000 claims abstract description 58
- 230000015654 memory Effects 0.000 claims description 64
- 238000004590 computer program Methods 0.000 claims description 24
- 238000012545 processing Methods 0.000 abstract description 14
- 238000010586 diagram Methods 0.000 description 15
- 238000005516 engineering process Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000000151 deposition Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000035800 maturation Effects 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- PEDCQBHIVMGVHV-UHFFFAOYSA-N Glycerine Chemical compound OCC(O)CO PEDCQBHIVMGVHV-UHFFFAOYSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- GOLXNESZZPUPJE-UHFFFAOYSA-N spiromesifen Chemical compound CC1=CC(C)=CC(C)=C1C(C(O1)=O)=C(OC(=O)CC(C)(C)C)C11CCCC1 GOLXNESZZPUPJE-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/221—Column-oriented storage; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24532—Query optimisation of parallel queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of data storage, querying method, devices, equipment, computer readable storage medium, belong to technical field of data processing.Date storage method based on Hbase includes: to obtain the corresponding Region mark of user data to be stored, user information and temporal information；The RowKey of user data is generated according to Region mark, user information and temporal information；It is identified according to Region and stores user data into corresponding Region.Data query method based on Hbase includes: to be grouped the RowKey of user data to be checked according to Region mark, RowKey includes that the corresponding Region mark of user data, user information and temporal information, every group of RowKey are identified comprising identical Region；Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, each thread executes the inquiry of one group of RowKey.The present invention can be improved the storage efficiency and search efficiency of data.
Description
Technical field
The present invention relates to technical field of data processing, a kind of data storage, querying method, device, equipment, meter are particularly related to
Calculation machine readable storage medium storing program for executing.
Background technique
With the rapid development of network technology, the rapidly growth of data volume, in order to analyze and utilize these huge data
Resource, traditional technology already encounter huge obstacle, can not be competent at the task of big data analysis.And in order to meet big data
The requirement of analysis, Google propose MapReduce technology, it is a kind of towards large-scale data analysis processing and parallel computation
Programming model.In technology needed for big data, distributed file system, distributed data base etc. are all to be suitable for big data
Technology.HBase is a kind of expansible, large-scale distributed database of support.It is using HadoopHDFS as it
Document storage system.Since it is with good expansibility, fault-tolerance and random reading capability, MapReduce is supported simultaneously
Row calculates, and is received by more and more companies.With the lasting improvement of HBase performance and stability, HBase is increasingly becoming
One of the standard in the field big data NoSQL.HBase is that storing data, table are made of row and column in the form of a table, if column are divided into
Dry column family (row family), RowKey (line unit) is the major key for retrieving record.
But existing HBase has the disadvantage in that
(1) meet real time sequence inquiry, but be unsatisfactory for quick magnanimity write-in.
It is write according to certain Sequentially continuous, such as according to time series, continuously designs continuous RowKey, HBase data are written
Library.Write in this way data volume it is small when perhaps have good readwrite performance, it is very big if it is data volume, then HBase
Just it is easy to appear hot issues, i.e. certain Regionserver (HBase cluster operates in the service on each working node)
Extension, and then hot spot is transferred to another Regionserver, and the Regionserver being then transferred hangs, continuous in this way to pass
Lead, until entire cluster hang, it is unavailable to there is HBase.
(2) meet quick magnanimity write-in, but be unsatisfactory for real time sequence inquiry.
The continuous sequence RowKey occurred in the short time is distributed to more according to certain existing Hash (Hash) algorithm
Regionserver writes.Due to having done Hash distribution here, then at the time of reading, it is necessary to merge the Region of this multichannel Hash
(domain).Obviously it reads and needs a large amount of retrieval, need a large amount of network I/O (input/output) and occupy a large amount of memory to merge
These values.This is worthless for needing the inquiry of real-time response.Even offline business, it is also difficult to endure.
In conclusion existing HBase is difficult to meet the no write de-lay and real-time query of mass data simultaneously.
Summary of the invention
It can the technical problem to be solved in the present invention is to provide a kind of data storage, querying method, device, equipment, computer
Storage medium is read, can be improved the storage efficiency and search efficiency of data.
In order to solve the above technical problems, the embodiment of the present invention offer technical solution is as follows:
On the one hand, a kind of date storage method based on Hbase is provided, comprising:
Obtain the corresponding Region mark of user data to be stored, user information and temporal information；
The RowKey of the user data is generated according to Region mark, user information and temporal information；
It is identified according to the Region and stores the user data into corresponding Region.
Further, obtaining the corresponding Region mark of user data to be stored includes:
The number N of Region is calculated according to the total amount of user data to be stored；
The Region is obtained to the user information complementation after carrying out cryptographic Hash calculating using N to identify.
Further, obtaining the corresponding user information of user data to be stored includes:
The user information is obtained after encrypting to the user identifier of the user data, the user identifier is uniquely marked
Know the corresponding user of the user data.
Further, obtaining the corresponding temporal information of user data to be stored includes:
Year information and month information to the user data obtain the temporal information after carrying out inverted sequence.
The data query method based on Hbase that the embodiment of the invention also provides a kind of, comprising:
The RowKey of user data to be checked is grouped according to Region mark, the RowKey includes number of users
According to corresponding Region mark, user information and temporal information, every group of RowKey is identified comprising identical Region；
Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, each thread executes looking into for one group of RowKey
It askes.
Further, the RowKey by user data to be checked is grouped according to Region mark includes:
When the number that a Region identifies corresponding RowKey is greater than threshold value, the Region is identified corresponding
RowKey points are at least two groups, so that the number of every group of RowKey is not more than threshold value.
Further, described to include: to multiple groups RowKey execution inquiry operation using concurrent multiple threads
When executing inquiry operation to one group of RowKey in a Region, one RowKey of inquiry obtains corresponding query result
Afterwards, which is stored in buffer area, and the seek method of Region is called to jump directly to the position of next RowKey
The inquiry for carrying out next RowKey is set, in one group of RowKey poll-final or full buffer area, is stored in return cache area
Query result.
The embodiment of the invention also provides a kind of data storage devices based on Hbase, comprising:
Data obtaining module, for obtaining the corresponding Region mark of user data to be stored, user information and time
Information；
RowKey generation module, for generating the user according to Region mark, user information and temporal information
The RowKey of data；
Data memory module stores the user data to corresponding Region for identifying according to the Region
In.
The embodiment of the invention also provides a kind of data query devices based on Hbase, comprising:
Grouping module, it is described for the RowKey of user data to be checked to be grouped according to Region mark
RowKey includes that the corresponding Region mark of user data, user information and temporal information, every group of RowKey include identical
Region mark；
Enquiry module, for executing inquiry operation to multiple groups RowKey using concurrent multiple threads, each thread is executed
The inquiry of one group of RowKey.
The embodiment of the invention also provides a kind of electronic equipments for realizing the date storage method based on Hbase, comprising:
Processor；With
Memory is stored with computer program instructions in the memory,
Wherein, when the computer program instructions are run by the processor, so that the processor executes following step
It is rapid:
Obtain the corresponding Region mark of user data to be stored, user information and temporal information；
The RowKey of the user data is generated according to Region mark, user information and temporal information；
It is identified according to the Region and stores the user data into corresponding Region.
The embodiment of the invention also provides a kind of electronic equipments for realizing the data query method based on Hbase, comprising:
Processor；With
Memory is stored with computer program instructions in the memory,
Wherein, when the computer program instructions are run by the processor, so that the processor executes following step
It is rapid:
The RowKey of user data to be checked is grouped according to Region mark, the RowKey includes number of users
According to corresponding Region mark, user information and temporal information, every group of RowKey is identified comprising identical Region；
Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, each thread executes looking into for one group of RowKey
It askes.
The embodiment of the invention also provides a kind of computer readable storage medium, the computer-readable recording medium storage
There is computer program, when the computer program is run by processor, so that the processor executes following steps:
Obtain the corresponding Region mark of user data to be stored, user information and temporal information；
The RowKey of the user data is generated according to Region mark, user information and temporal information；
It is identified according to the Region and stores the user data into corresponding Region.
The embodiment of the invention also provides a kind of computer readable storage medium, the computer-readable recording medium storage
There is computer program, when the computer program is run by processor, so that the processor executes following steps:
The RowKey of user data to be checked is grouped according to Region mark, the RowKey includes number of users
According to corresponding Region mark, user information and temporal information, every group of RowKey is identified comprising identical Region；
Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, each thread executes looking into for one group of RowKey
It askes.
The embodiment of the present invention has the advantages that
In above scheme, Rowkey is the major key of every record in Hbase database, and Rowkey of the invention includes
Region mark, user information and temporal information three parts, and identified according to Region and store user data to corresponding
In Region, it can make in user data equilibrium storage to every Region in this way, and can guarantee the use of same user
User data is stored in same Region.When inquiring user data, it is accurate fixed to be identified according to the Region in Rowkey
The position of the Region of position storage user data, improves search efficiency, and the RowKey for the user data that can will be inquired
It is grouped according to Region mark, inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, is further increased
Search efficiency.
Detailed description of the invention
Fig. 1 is the flow diagram of date storage method of the embodiment of the present invention based on Hbase；
Fig. 2 is the flow diagram that the embodiment of the present invention obtains the corresponding Region mark of user data to be stored；
Fig. 3 is the flow diagram that the embodiment of the present invention obtains the corresponding user information of user data to be stored；
Fig. 4 is the flow diagram that the embodiment of the present invention obtains the corresponding temporal information of user data to be stored；
Fig. 5 is the flow diagram of data query method of the embodiment of the present invention based on Hbase；
Fig. 6 is that the RowKey of user data to be checked is identified the stream being grouped by the embodiment of the present invention according to Region
Journey schematic diagram；
Fig. 7 is that the embodiment of the present invention illustrates the multiple groups RowKey process for executing inquiry operation using concurrent multiple threads
Figure；
Fig. 8 is the structural block diagram of data storage device of the embodiment of the present invention based on Hbase；
Fig. 9 is the structural block diagram of data query device of the embodiment of the present invention based on Hbase；
Figure 10 is the structural block diagram for the electronic equipment that the embodiment of the present invention realizes the date storage method based on Hbase；
Figure 11 is the structural block diagram for the electronic equipment that the embodiment of the present invention realizes the data query method based on Hbase；
Figure 12 is the flow diagram of data of embodiment of the present invention storage and data query；
Figure 13 is that the embodiment of the present invention is grouped the schematic diagram for executing inquiry operation to RowKey；
Figure 14 is the schematic diagram that the embodiment of the present invention is inquired using middleware.
Specific embodiment
To keep the embodiment of the present invention technical problems to be solved, technical solution and advantage clearer, below in conjunction with
Drawings and the specific embodiments are described in detail.
The embodiment of the present invention provides a kind of data storage, querying method, device, equipment, computer readable storage medium,
It can be improved the storage efficiency and search efficiency of data.
Embodiment one
The embodiment of the present invention provides a kind of date storage method based on Hbase, as shown in Figure 1, comprising:
Step 101: obtaining the corresponding Region mark of user data to be stored, user information and temporal information；
Step 102: the user data is generated according to Region mark, user information and temporal information
RowKey；
Step 103: being identified according to the Region and store the user data into corresponding Region.
In the present embodiment, Rowkey is the major key of every record in Hbase database, and Rowkey of the invention includes
Region mark, user information and temporal information three parts, and identified according to Region and store user data to corresponding
In Region, it can make in user data equilibrium storage to every Region in this way, and can guarantee the use of same user
User data is stored in same Region.When inquiring user data, it is accurate fixed to be identified according to the Region in Rowkey
The position of the Region of position storage user data, improves search efficiency, and the RowKey for the user data that can will be inquired
It is grouped according to Region mark, inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, is further increased
Search efficiency.
As an example, as shown in Fig. 2, obtaining user data to be stored corresponding Region mark and including:
Step 1011: the number N of Region is calculated according to the total amount of user data to be stored；
Step 1012: the Region being obtained to the user information complementation after carrying out cryptographic Hash calculating using N and is identified.
As an example, as shown in figure 3, obtaining the corresponding user information of user data to be stored and including:
Step 1013: the user information, the user are obtained after encrypting to the user identifier of the user data
Identify the corresponding user of user data described in unique identification.
As an example, as shown in figure 4, obtaining the corresponding temporal information of user data to be stored and including:
Step 1014: year information and month information to the user data obtain the time letter after carrying out inverted sequence
Breath.
Embodiment two
The data query method based on Hbase that the embodiment of the invention also provides a kind of, as shown in Figure 5, comprising:
Step 201: the RowKey of user data to be checked being grouped according to Region mark, the RowKey packet
The corresponding Region mark of user data, user information and temporal information, every group of RowKey is included to identify comprising identical Region；
Step 202: inquiry operation being executed to multiple groups RowKey using concurrent multiple threads, each thread executes one group
The inquiry of RowKey.
In the present embodiment, Rowkey includes Region mark, user information and temporal information three parts, in inquiry number of users
According to when, can according to the Region in Rowkey identify be accurately positioned storage user data Region position, improve inquiry
Efficiency, and the RowKey for the user data that can will be inquired is grouped according to Region mark, using concurrent multiple
Thread executes inquiry operation to multiple groups RowKey, further increases search efficiency.
As an example, as shown in fig. 6, the RowKey by user data to be checked according to Region identify into
Row is grouped
Step 2011: when the number that a Region identifies corresponding RowKey is greater than threshold value, the Region being identified
Corresponding RowKey points are at least two groups, so that the number of every group of RowKey is not more than threshold value.
As an example, as shown in fig. 7, described execute inquiry operation to multiple groups RowKey using concurrent multiple threads
Include:
Step 2021: when executing inquiry operation to one group of RowKey in a Region, one RowKey of inquiry is corresponded to
Query result after, which is stored in buffer area, and it is next to call the seek method of Region to jump directly to
The position of RowKey carries out the inquiry of next RowKey, in one group of RowKey poll-final or full buffer area, return cache
The query result stored in area.
Embodiment three
The embodiment of the invention also provides a kind of data storage devices based on Hbase, as shown in Figure 8, comprising:
Data obtaining module 11, for obtain the corresponding Region mark of user data wait store, user information and when
Between information；
RowKey generation module 12, for generating the use according to Region mark, user information and temporal information
The RowKey of user data；
Data memory module 13 stores the user data to corresponding Region for identifying according to the Region
In.
In the present embodiment, Rowkey is the major key of every record in Hbase database, and Rowkey of the invention includes
Region mark, user information and temporal information three parts, and identified according to Region and store user data to corresponding
In Region, it can make in user data equilibrium storage to every Region in this way, and can guarantee the use of same user
User data is stored in same Region.When inquiring user data, it is accurate fixed to be identified according to the Region in Rowkey
The position of the Region of position storage user data, improves search efficiency, and the RowKey for the user data that can will be inquired
It is grouped according to Region mark, inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, is further increased
Search efficiency.
As an example, data obtaining module 11 is specifically used for being calculated according to the total amount of user data to be stored
The number N of Region；The Region is obtained to the user information complementation after carrying out cryptographic Hash calculating using N to identify.
As an example, data obtaining module 11 is specifically used for after encrypting to the user identifier of the user data
The user information is obtained, the corresponding user of user data described in the user identifier unique identification.
As an example, data obtaining module 11 is specifically used for year information and month information to the user data
The temporal information is obtained after carrying out inverted sequence.
Example IV
The embodiment of the invention also provides a kind of data query devices based on Hbase, as shown in Figure 9, comprising:
Grouping module 21, it is described for the RowKey of user data to be checked to be grouped according to Region mark
RowKey includes that the corresponding Region mark of user data, user information and temporal information, every group of RowKey include identical
Region mark；
Enquiry module 22, for executing inquiry operation to multiple groups RowKey using concurrent multiple threads, each thread is held
The inquiry of one group of RowKey of row.
In the present embodiment, Rowkey includes Region mark, user information and temporal information three parts, in inquiry number of users
According to when, can according to the Region in Rowkey identify be accurately positioned storage user data Region position, improve inquiry
Efficiency, and the RowKey for the user data that can will be inquired is grouped according to Region mark, using concurrent multiple
Thread executes inquiry operation to multiple groups RowKey, further increases search efficiency.
As an example, grouping module 21, will when the number that a Region identifies corresponding RowKey is greater than threshold value
It is at least two groups that the Region, which identifies corresponding RowKey points, so that the number of every group of RowKey is not more than threshold value.
As an example, when enquiry module 22 executes inquiry operation to one group of RowKey in a Region, inquiry one
After RowKey obtains corresponding query result, which is stored in buffer area, and calls the seek method of Region
The position for jumping directly to next RowKey carries out the inquiry of next RowKey, in one group of RowKey poll-final or buffer area
Man Shi, the query result stored in return cache area.
Embodiment five
The embodiment of the invention also provides a kind of electronic equipments 30 for realizing the date storage method based on Hbase, such as scheme
Shown in 10, comprising:
Processor 32；With
Memory 34 is stored with computer program instructions in the memory 34,
Wherein, when the computer program instructions are run by the processor so that the processor 32 execute it is following
Step:
Obtain the corresponding Region mark of user data to be stored, user information and temporal information；
The RowKey of the user data is generated according to Region mark, user information and temporal information；
It is identified according to the Region and stores the user data into corresponding Region.
Further, as shown in Figure 10, the electronic equipment 30 for realizing the date storage method based on Hbase further includes network
Interface 31, input equipment 33, hard disk 35 and display equipment 36.
It can be interconnected by bus architecture between above-mentioned each interface and equipment.It may include any that bus architecture, which can be,
The bus and bridge of the interconnection of quantity.One or more central processing unit (CPU) specifically represented by processor 32, and by depositing
The various circuit connections for one or more memory that reservoir 34 represents are together.Bus architecture can also such as will be set periphery
Standby, voltage-stablizer is together with the various other circuit connections of management circuit or the like.It is appreciated that bus architecture is for real
Connection communication between these existing components.Bus architecture except include data/address bus in addition to, further include power bus, control bus and
Status signal bus in addition, these are all it is known in the art, therefore is no longer described in greater detail herein.
The network interface 31 can connect to network (such as internet, local area network), dependency number obtained from network
According to, such as user data to be stored, and can be stored in hard disk 35.
The input equipment 33, can receive the various instructions of operator's input, and be sent to processor 32 for holding
Row.The input equipment 33 may include keyboard or pointing device (for example, mouse, trace ball (trackball), touch-sensitive plate
Or touch screen etc..
The display equipment 36, the result that processor 32 executes instruction acquisition can be shown.
The memory 34, program necessary to running for storage program area and data and processor 32 calculate
The data such as intermediate result in the process.
It is appreciated that the memory 34 in the embodiment of the present invention can be volatile memory or nonvolatile memory,
It or may include both volatile and non-volatile memories.Wherein, nonvolatile memory can be read-only memory (ROM),
Programmable read only memory (PROM), Erasable Programmable Read Only Memory EPROM (EPROM), electrically erasable programmable read-only memory
(EEPROM) or flash memory.Volatile memory can be random access memory (RAM), be used as External Cache.Herein
The memory 34 of the device and method of description is intended to include but is not limited to the memory of these and any other suitable type.
In some embodiments, memory 34 stores following element, executable modules or data structures, or
Their subset or their superset: operating system 341 and application program 342.
Wherein, operating system 341 include various system programs, such as ccf layer, core library layer, driving layer etc., for real
Existing various basic businesses and the hardware based task of processing.Application program 342 includes various application programs, such as browser
(Browser) etc., for realizing various applied business.Realize that the program of present invention method may be embodied in application program
In 342.
Above-mentioned processor 32, when calling and execute the application program and data that are stored in the memory 34, specifically,
When can be the program or instruction that store in application program 342, the corresponding Region mark of available user data to be stored
Knowledge, user information and temporal information；The user data is generated according to Region mark, user information and temporal information
RowKey；It is identified according to the Region and stores the user data into corresponding Region.
The method that the above embodiment of the present invention discloses can be applied in processor 32, or be realized by processor 32.Place
Managing device 32 may be a kind of IC chip, the processing capacity with signal.During realization, each step of the above method
It can be completed by the integrated logic circuit of the hardware in processor 32 or the instruction of software form.Above-mentioned processor 32 can
To be general processor, digital signal processor (DSP), specific integrated circuit (ASIC), ready-made programmable gate array (FPGA)
Either perhaps transistor logic, discrete hardware components may be implemented or execute for other programmable logic device, discrete gate
Disclosed each method, step and logic diagram in the embodiment of the present invention.General processor can be microprocessor or this at
Reason device is also possible to any conventional processor etc..The step of method in conjunction with disclosed in the embodiment of the present invention, can directly embody
Execute completion for hardware decoding processor, or in decoding processor hardware and software module combination execute completion.Software
Module can be located at random access memory, flash memory, read-only memory, programmable read only memory or electrically erasable programmable storage
In the storage medium of this fields such as device, register maturation.The storage medium is located at memory 34, and processor 32 reads memory 34
In information, in conjunction with its hardware complete the above method the step of.
It is understood that embodiments described herein can with hardware, software, firmware, middleware, microcode or its
Combination is to realize.For hardware realization, processing unit be may be implemented in one or more specific integrated circuits (ASIC), number letter
Number processor DSP), digital signal processing appts (DSPD), programmable logic device (PLD), field programmable gate array
(FPGA), general processor, controller, microcontroller, microprocessor, other electronics lists for executing herein described function
In member or combinations thereof.
For software implementations, it can be realized herein by executing the module (such as process, function etc.) of function described herein
The technology.Software code is storable in memory and is executed by processor.Memory can in the processor or
It is realized outside processor.
Specifically, processor 32 calculates the number N of Region according to the total amount of user data to be stored；Using N into
User information complementation after row cryptographic Hash calculates obtains the Region mark.
Specifically, processor 32 obtains the user information, institute after encrypting to the user identifier of the user data
State the corresponding user of user data described in user identifier unique identification.
Specifically, when processor 32 is to obtaining described after year information and month information the progress inverted sequence of the user data
Between information.
In the present embodiment, Rowkey is the major key of every record in Hbase database, and Rowkey of the invention includes
Region mark, user information and temporal information three parts, and identified according to Region and store user data to corresponding
In Region, it can make in user data equilibrium storage to every Region in this way, and can guarantee the use of same user
User data is stored in same Region.When inquiring user data, it is accurate fixed to be identified according to the Region in Rowkey
The position of the Region of position storage user data, improves search efficiency, and the RowKey for the user data that can will be inquired
It is grouped according to Region mark, inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, is further increased
Search efficiency.
Embodiment six
The embodiment of the invention also provides a kind of electronic equipments 40 for realizing the data query method based on Hbase, such as scheme
Shown in 11, comprising:
Processor 42；With
Memory 44 is stored with computer program instructions in the memory 44,
Wherein, when the computer program instructions are run by the processor so that the processor 42 execute it is following
Step:
The RowKey of user data to be checked is grouped according to Region mark, the RowKey includes number of users
According to corresponding Region mark, user information and temporal information, every group of RowKey is identified comprising identical Region；
Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, each thread executes looking into for one group of RowKey
It askes.
Further, as shown in figure 11, the electronic equipment 40 for realizing the data query method based on Hbase further includes network
Interface 41, input equipment 43, hard disk 45 and display equipment 46.
It can be interconnected by bus architecture between above-mentioned each interface and equipment.It may include any that bus architecture, which can be,
The bus and bridge of the interconnection of quantity.One or more central processing unit (CPU) specifically represented by processor 42, and by depositing
The various circuit connections for one or more memory that reservoir 44 represents are together.Bus architecture can also such as will be set periphery
Standby, voltage-stablizer is together with the various other circuit connections of management circuit or the like.It is appreciated that bus architecture is for real
Connection communication between these existing components.Bus architecture except include data/address bus in addition to, further include power bus, control bus and
Status signal bus in addition, these are all it is known in the art, therefore is no longer described in greater detail herein.
The network interface 41 can connect to network (such as internet, local area network), dependency number obtained from network
According to, such as the RowKey of user data to be checked, and can be stored in hard disk 45.
The input equipment 43, can receive the various instructions of operator's input, and be sent to processor 42 for holding
Row.The input equipment 43 may include keyboard or pointing device (for example, mouse, trace ball (trackball), touch-sensitive plate
Or touch screen etc..
The display equipment 46, the result that processor 42 executes instruction acquisition can be shown.
The memory 44, program necessary to running for storage program area and data and processor 42 calculate
The data such as intermediate result in the process.
It is appreciated that the memory 44 in the embodiment of the present invention can be volatile memory or nonvolatile memory,
It or may include both volatile and non-volatile memories.Wherein, nonvolatile memory can be read-only memory (ROM),
Programmable read only memory (PROM), Erasable Programmable Read Only Memory EPROM (EPROM), electrically erasable programmable read-only memory
(EEPROM) or flash memory.Volatile memory can be random access memory (RAM), be used as External Cache.Herein
The memory 44 of the device and method of description is intended to include but is not limited to the memory of these and any other suitable type.
In some embodiments, memory 44 stores following element, executable modules or data structures, or
Their subset or their superset: operating system 441 and application program 442.
Wherein, operating system 441 include various system programs, such as ccf layer, core library layer, driving layer etc., for real
Existing various basic businesses and the hardware based task of processing.Application program 442 includes various application programs, such as browser
(Browser) etc., for realizing various applied business.Realize that the program of present invention method may be embodied in application program
In 442.
Above-mentioned processor 42, when calling and execute the application program and data that are stored in the memory 44, specifically,
When can be the program or instruction that store in application program 442, can by the RowKey of user data to be checked according to
Region mark is grouped, and the RowKey includes user data corresponding Region mark, user information and temporal information,
Every group of RowKey is identified comprising identical Region；Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, often
One thread executes the inquiry of one group of RowKey.
The method that the above embodiment of the present invention discloses can be applied in processor 42, or be realized by processor 42.Place
Managing device 42 may be a kind of IC chip, the processing capacity with signal.During realization, each step of the above method
It can be completed by the integrated logic circuit of the hardware in processor 42 or the instruction of software form.Above-mentioned processor 42 can
To be general processor, digital signal processor (DSP), specific integrated circuit (ASIC), ready-made programmable gate array (FPGA)
Either perhaps transistor logic, discrete hardware components may be implemented or execute for other programmable logic device, discrete gate
Disclosed each method, step and logic diagram in the embodiment of the present invention.General processor can be microprocessor or this at
Reason device is also possible to any conventional processor etc..The step of method in conjunction with disclosed in the embodiment of the present invention, can directly embody
Execute completion for hardware decoding processor, or in decoding processor hardware and software module combination execute completion.Software
Module can be located at random access memory, flash memory, read-only memory, programmable read only memory or electrically erasable programmable storage
In the storage medium of this fields such as device, register maturation.The storage medium is located at memory 44, and processor 42 reads memory 44
In information, in conjunction with its hardware complete the above method the step of.
It is understood that embodiments described herein can with hardware, software, firmware, middleware, microcode or its
Combination is to realize.For hardware realization, processing unit be may be implemented in one or more specific integrated circuits (ASIC), number letter
Number processor DSP), digital signal processing appts (DSPD), programmable logic device (PLD), field programmable gate array
(FPGA), general processor, controller, microcontroller, microprocessor, other electronics lists for executing herein described function
In member or combinations thereof.
For software implementations, it can be realized herein by executing the module (such as process, function etc.) of function described herein
The technology.Software code is storable in memory and is executed by processor.Memory can in the processor or
It is realized outside processor.
Specifically, processor 42, will be described when the number that a Region identifies corresponding RowKey is greater than threshold value
It is at least two groups that Region, which identifies corresponding RowKey points, so that the number of every group of RowKey is not more than threshold value.
Specifically, when processor 42 executes inquiry operation to one group of RowKey in a Region, one RowKey of inquiry is obtained
To after corresponding query result, which is stored in buffer area, and the seek method of Region is called directly to jump
The inquiry that next RowKey is carried out to the position of next RowKey is returned in one group of RowKey poll-final or full buffer area
Return the query result stored in buffer area.
In the present embodiment, Rowkey includes Region mark, user information and temporal information three parts, in inquiry number of users
According to when, can according to the Region in Rowkey identify be accurately positioned storage user data Region position, improve inquiry
Efficiency, and the RowKey for the user data that can will be inquired is grouped according to Region mark, using concurrent multiple
Thread executes inquiry operation to multiple groups RowKey, further increases search efficiency.
Embodiment seven
The embodiment of the invention also provides a kind of computer readable storage medium, the computer-readable recording medium storage
There is computer program, when the computer program is run by processor, so that the processor executes following steps:
Obtain the corresponding Region mark of user data to be stored, user information and temporal information；
The RowKey of the user data is generated according to Region mark, user information and temporal information；
It is identified according to the Region and stores the user data into corresponding Region.
Embodiment eight
The embodiment of the invention also provides a kind of computer readable storage medium, the computer-readable recording medium storage
There is computer program, when the computer program is run by processor, so that the processor executes following steps:
The RowKey of user data to be checked is grouped according to Region mark, the RowKey includes number of users
According to corresponding Region mark, user information and temporal information, every group of RowKey is identified comprising identical Region；
Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, each thread executes looking into for one group of RowKey
It askes.
Embodiment nine
Below by taking user data to be stored is user's geographic position data as an example, data of the invention are deposited in conjunction with attached drawing
Method for storing and querying method are described further, as shown in figure 12, the embodiment of the present invention specifically includes the following steps:
Step 301: the number N of Region is calculated according to user's geographic position data；
In the present embodiment, user data to be stored is the time series data of user's geographical location information, the leap of total data
Time can be 1 year, and it may include the timing of the geographical location information of all users of the whole network that data break, which can be 1~30 second,
Data.The number N of Region is calculated according to the total amount of data of user's geographic position data and the amount of storage of every Region first,
The number N of Region is preferably the integral multiple of RegionServer, is conducive to HBase in this way using MapReduce and carries out data
Processing.
In addition, the present embodiment also needs to forbid the automatic splitting function of Region, because Region is frequently divided automatically
Have many problems, such as make storage performance decline etc..
Step 302: user's geographic position data is stored in N number of Region；
It needs first to design table before storing user data.Rowkey is the major key of every record in Hbase database,
Row in table is ranked up according to RowKey, and user data sorts according to the syllable sequence of RowKey and stores, the access of all pairs of tables
RowKey will be passed through.Therefore the design of RowKey will directly affect the efficiency and data balancing of data storage, and influence
The efficiency of inquiry.
In the present embodiment, Rowkey is consisted of three parts: Region ID (mark), user information and temporal information.I.e.
Rowkey=Region_id (4)+(user_id) (32)+Date (6).
Wherein, Region_id, that is, Region ID can occupy the space of 4 bytes, and user_id, that is, user information can account for
With the space of 32 bytes, Date, that is, temporal information can occupy the space of 6 bytes.
Region is the minimum unit of distributed storage and load balancing in Hbase, can accurately be determined by Region ID
The position of position Region.Region ID can be acquired by following formula:
Region_id=Hash (user_id) %Num (Region)；
Wherein, Hash (user_id) is that cryptographic Hash calculating is carried out to user information, and Num (Region) is that step 301 is fallen into a trap
The number of the Region of calculating.It can be assigned to user data is balanced on each Region by this kind of calculation, and
And when inquiring, the position of Region can be accurately positioned according to Region ID.
User identifier being capable of one user of unique identification, it is ensured that the uniqueness of RowKey, user identifier can be for user'ss
Cell-phone number, identification card number etc..Information leakage needs are encrypted to obtain by using modes such as md5 to user identifier in order to prevent
User information.
Temporal information identifies the creation time of user data and sorts according to the inverted sequence of time, the time of the present embodiment
Information is made of year and the moon, and such design largely reduces the line numbers of data in mass data, if with the date or
Person's date Hour Minute Second carrys out makeup time information, and the quantity of the RowKey generated in this way has 30 times to 2592000 times of increase.
One month user data is marked using a RowKey, and carries out inverted sequence, i.e. Date=999999- to the time
Yyyymm, such as: user data creation time is 03 month 2017, and the corresponding inverted sequence time is exactly 999999-201703=
798296.It can be with quick search to nearest data according to the inverted sequence sequence of time.
In one specific example, user_id=13538279983, Num (Region)=50, time=201702, then finally
RowKey=00407D2D0BC12BFD63155F309A2FE6167D8E_798297, wherein 0040 is Region ID,
7D2D0BC12BFD63155F309A2FE6167D8E is to the user information obtained after user identifier encryption, and 798297 be the time
Information.
In the column cluster design of table, use date can control as column name, such columns.And the date can also be with
The data of scheduled date are quickly inquired as secondary index user.
In the present embodiment, setting Hbase can store all historical datas, each version corresponds to a timestamp.
Such as:
Wherein, Column represents day, and Timestamp is timestamp, and Val is geographical location (including longitude and the latitude of user
Degree), it can be seen that multiple Rowkey are stored in a table, and each timestamp every day is stored under each Rowkey and is corresponded to
Customer position information.
When storing user's geographic position data, because of RowKey included Region ID, according to Region
ID is stored data on corresponding Region, and it is same can to guarantee that the data storage of identical Region ID has been arrived in this way
On Region, and the data of same user are also stored on the same Region.
Step 303: the RowKey of user data to be checked is grouped according to Region mark；
According to step 301-302 creation database in retrieve data when, as shown in figure 13, first will client it is defeated
The RowKey of the data to be checked entered is grouped according to Region ID, wherein the first four byte representation Region of Rowkey
ID。
Equilibrium is needed when inquiring data, if the quantity of one group of RowKey is too many, needs one group of RowKey splitting into two
Group RowKey or more multiple groups RowKey, to guarantee the equilibrium of inquiry.
Rowkey is ranked up according to ASCII after grouping.
Step 304: inquiry operation being executed to multiple groups RowKey using concurrent multiple threads, each thread executes one group
The inquiry of RowKey；
Setting inquiry concurrent thread number is configured according to system environments first, such as: server (CPUCore:12,
RAM:256G, Disk > 10T), System >=Centos6.5, it is proposed that the Thread Count of setting is not higher than 48 threads.
Per thread executes the inquiry of one group of RowKey, i.e. per thread only needs to be inquired on a Region,
And multiple threads concurrently execute inquiry operation, and the efficiency of data query can be improved.If the group number L of RowKey is greater than thread
Number K after needing that K thread is waited to complete inquiry, then is executed then after distributing corresponding thread one by one for K group RowKey
The inquiry operation of remaining (L-K) group RowKey.
When carrying out data query in a Region, batch query can be carried out by middleware, can be improved in this way
Search efficiency reduces the communication frequency with client.
As shown in figure 14, increase middleware on Region, the process of inquiry is controlled by middleware, in batch query
During one group of RowKey, after obtaining the corresponding query result of a RowKey, the result of inquiry is stored to slow first
It deposits in area, and calls the seek method of Region to jump directly to the position of next RowKey, and carry out next RowKey
Inquiry, until poll-final or buffer area are full, then return result to client..
Step 305: returning to query result.
The corresponding user data of multiple groups RowKey is returned to client.Wherein, query result can be as shown in the table,
In, time range is shown in the column of data, and the data of continuous time period can be obtained according to version.
| Column=cf:9798 | Timestamp=1478100960000 | Val=118.095,24.5834 |
| Column=cf:9798 | Timestamp=1478090320000 | Val=118.094,24.5830 |
| Column=cf:9798 | Timestamp=1478080360000 | Val=118.095,24.5835 |
| Column=cf:9798 | Timestamp=1478070450000 | Val=118.096,24.5836 |
The Rowkey of the present embodiment includes Region mark, user information and temporal information three parts, and according to Region
Mark stores user data into corresponding Region, can make user data equilibrium storage to every Region in this way
On, and can guarantee that the user data of same user is stored in same Region.It, can basis when inquiring user data
The position of the Region of storage user data is accurately positioned in Region mark in Rowkey, improves search efficiency, and can be with
The RowKey for the user data that will be inquired is grouped according to Region mark, using concurrent multiple threads to multiple groups
RowKey executes inquiry operation, further increases search efficiency.
It is the preferred embodiment of the present invention above, it is noted that for those skilled in the art,
Without departing from the principles of the present invention, it can also make several improvements and retouch, these improvements and modifications should also regard
For protection scope of the present invention.
Claims (13)
1. a kind of date storage method based on Hbase characterized by comprising
Obtain the corresponding Region mark of user data to be stored, user information and temporal information；
The RowKey of the user data is generated according to Region mark, user information and temporal information；
It is identified according to the Region and stores the user data into corresponding Region.
2. the date storage method according to claim 1 based on Hbase, which is characterized in that obtain user to be stored
The corresponding Region of data, which is identified, includes:
The number N of Region is calculated according to the total amount of user data to be stored；
The Region is obtained to the user information complementation after carrying out cryptographic Hash calculating using N to identify.
3. the date storage method according to claim 1 based on Hbase, which is characterized in that obtain user to be stored
The corresponding user information of data includes:
The user information, the user identifier unique identification institute are obtained after encrypting to the user identifier of the user data
State the corresponding user of user data.
4. the date storage method according to claim 1 based on Hbase, which is characterized in that obtain user to be stored
The corresponding temporal information of data includes:
Year information and month information to the user data obtain the temporal information after carrying out inverted sequence.
5. a kind of data query method based on Hbase characterized by comprising
The RowKey of user data to be checked is grouped according to Region mark, the RowKey includes user data pair
Region mark, user information and the temporal information answered, every group of RowKey are identified comprising identical Region；
Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, each thread executes the inquiry of one group of RowKey.
6. the data query method according to claim 5 based on Hbase, which is characterized in that described by use to be checked
The RowKey of user data is grouped according to Region mark
When the number that a Region identifies corresponding RowKey is greater than threshold value, the Region is identified RowKey points corresponding
For at least two groups, so that the number of every group of RowKey is not more than threshold value.
7. the data query method according to claim 5 based on Hbase, which is characterized in that described to utilize concurrently more
A thread executes inquiry operation to multiple groups RowKey
When executing inquiry operation to one group of RowKey in a Region, after one RowKey of inquiry obtains corresponding query result,
The query result is stored in buffer area, and call the seek method of Region jump directly to the position of next RowKey into
The inquiry of the next RowKey of row, in one group of RowKey poll-final or full buffer area, the inquiry that is stored in return cache area
As a result.
8. a kind of data storage device based on Hbase characterized by comprising
Data obtaining module, for obtaining the corresponding Region mark of user data to be stored, user information and temporal information；
RowKey generation module, for generating the user data according to Region mark, user information and temporal information
RowKey；
Data memory module stores the user data into corresponding Region for identifying according to the Region.
9. a kind of data query device based on Hbase characterized by comprising
Grouping module, for the RowKey of user data to be checked to be grouped according to Region mark, the RowKey
It is marked including the corresponding Region mark of user data, user information and temporal information, every group of RowKey comprising identical Region
Know；
Enquiry module, for executing inquiry operation to multiple groups RowKey using concurrent multiple threads, each thread executes one group
The inquiry of RowKey.
10. a kind of electronic equipment for realizing the date storage method based on Hbase characterized by comprising
Processor；With
Memory is stored with computer program instructions in the memory,
Wherein, when the computer program instructions are run by the processor, so that the processor executes following steps:
Obtain the corresponding Region mark of user data to be stored, user information and temporal information；
The RowKey of the user data is generated according to Region mark, user information and temporal information；
It is identified according to the Region and stores the user data into corresponding Region.
11. a kind of electronic equipment for realizing the data query method based on Hbase characterized by comprising
Processor；With
Memory is stored with computer program instructions in the memory,
Wherein, when the computer program instructions are run by the processor, so that the processor executes following steps:
The RowKey of user data to be checked is grouped according to Region mark, the RowKey includes user data pair
Region mark, user information and the temporal information answered, every group of RowKey are identified comprising identical Region；
Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, each thread executes the inquiry of one group of RowKey.
12. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists
In when the computer program is run by processor, so that the processor executes following steps:
Obtain the corresponding Region mark of user data to be stored, user information and temporal information；
The RowKey of the user data is generated according to Region mark, user information and temporal information；
It is identified according to the Region and stores the user data into corresponding Region.
13. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists
In when the computer program is run by processor, so that the processor executes following steps:
The RowKey of user data to be checked is grouped according to Region mark, the RowKey includes user data pair
Region mark, user information and the temporal information answered, every group of RowKey are identified comprising identical Region；
Inquiry operation is executed to multiple groups RowKey using concurrent multiple threads, each thread executes the inquiry of one group of RowKey.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710911189.3A CN110019199A (en) | 2017-09-29 | 2017-09-29 | Data storage, querying method, device, equipment, computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710911189.3A CN110019199A (en) | 2017-09-29 | 2017-09-29 | Data storage, querying method, device, equipment, computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110019199A true CN110019199A (en) | 2019-07-16 |
Family
ID=67186463
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710911189.3A Pending CN110019199A (en) | 2017-09-29 | 2017-09-29 | Data storage, querying method, device, equipment, computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019199A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109213758A (en) * | 2018-07-24 | 2019-01-15 | 中国联合网络通信集团有限公司 | Data access method, device, equipment and computer readable storage medium |
| CN110442581A (en) * | 2019-08-05 | 2019-11-12 | 北京智慧远景科技产业有限公司 | A kind of method and system that data are stored, read |
| CN110597922A (en) * | 2019-09-27 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Data processing method, device, terminal and storage medium |
| CN110688382A (en) * | 2019-09-25 | 2020-01-14 | 深圳市赛为智能股份有限公司 | Data storage query method and device, computer equipment and storage medium |
| CN113127474A (en) * | 2019-12-30 | 2021-07-16 | 中国移动通信集团四川有限公司 | Data storage method and device and electronic equipment |
| CN113312383A (en) * | 2021-06-01 | 2021-08-27 | 拉卡拉支付股份有限公司 | Data query method, data query device, electronic equipment, storage medium and program product |
| CN113407518A (en) * | 2021-06-30 | 2021-09-17 | 中国银行股份有限公司 | Rowkey design method and device of Hbase database |
| CN113626490A (en) * | 2020-05-08 | 2021-11-09 | 杭州海康威视数字技术股份有限公司 | Data query method, device and equipment and storage medium |
| CN114328583A (en) * | 2021-12-27 | 2022-04-12 | 中科星图股份有限公司 | Real-time data query acceleration method for time sequence database |
| CN115185723A (en) * | 2022-06-02 | 2022-10-14 | 广州市玄武无线科技股份有限公司 | Method, system, device and storage medium for repairing Region meta-information |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103646073A (en) * | 2013-12-11 | 2014-03-19 | 浪潮电子信息产业股份有限公司 | Condition query optimizing method based on HBase table |
| CN105138592A (en) * | 2015-07-31 | 2015-12-09 | 武汉虹信技术服务有限责任公司 | Distributed framework-based log data storing and retrieving method |
| CN106777387A (en) * | 2017-02-16 | 2017-05-31 | 江苏海平面数据科技有限公司 | A kind of Internet of Things big data access method based on HBase |
-
2017
- 2017-09-29 CN CN201710911189.3A patent/CN110019199A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103646073A (en) * | 2013-12-11 | 2014-03-19 | 浪潮电子信息产业股份有限公司 | Condition query optimizing method based on HBase table |
| CN105138592A (en) * | 2015-07-31 | 2015-12-09 | 武汉虹信技术服务有限责任公司 | Distributed framework-based log data storing and retrieving method |
| CN106777387A (en) * | 2017-02-16 | 2017-05-31 | 江苏海平面数据科技有限公司 | A kind of Internet of Things big data access method based on HBase |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109213758B (en) * | 2018-07-24 | 2021-03-30 | 中国联合网络通信集团有限公司 | Data access method, device, equipment and computer readable storage medium |
| CN109213758A (en) * | 2018-07-24 | 2019-01-15 | 中国联合网络通信集团有限公司 | Data access method, device, equipment and computer readable storage medium |
| CN110442581A (en) * | 2019-08-05 | 2019-11-12 | 北京智慧远景科技产业有限公司 | A kind of method and system that data are stored, read |
| CN110688382B (en) * | 2019-09-25 | 2022-07-01 | 深圳市赛为智能股份有限公司 | Data storage query method and device, computer equipment and storage medium |
| CN110688382A (en) * | 2019-09-25 | 2020-01-14 | 深圳市赛为智能股份有限公司 | Data storage query method and device, computer equipment and storage medium |
| CN110597922A (en) * | 2019-09-27 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Data processing method, device, terminal and storage medium |
| CN110597922B (en) * | 2019-09-27 | 2024-05-14 | 腾讯科技（深圳）有限公司 | Data processing method, device, terminal and storage medium |
| CN113127474A (en) * | 2019-12-30 | 2021-07-16 | 中国移动通信集团四川有限公司 | Data storage method and device and electronic equipment |
| CN113626490A (en) * | 2020-05-08 | 2021-11-09 | 杭州海康威视数字技术股份有限公司 | Data query method, device and equipment and storage medium |
| CN113626490B (en) * | 2020-05-08 | 2023-08-25 | 杭州海康威视数字技术股份有限公司 | Data query method, device and equipment and storage medium |
| CN113312383A (en) * | 2021-06-01 | 2021-08-27 | 拉卡拉支付股份有限公司 | Data query method, data query device, electronic equipment, storage medium and program product |
| CN113407518B (en) * | 2021-06-30 | 2024-02-23 | 中国银行股份有限公司 | Rowkey design method and device of Hbase database |
| CN113407518A (en) * | 2021-06-30 | 2021-09-17 | 中国银行股份有限公司 | Rowkey design method and device of Hbase database |
| CN114328583A (en) * | 2021-12-27 | 2022-04-12 | 中科星图股份有限公司 | Real-time data query acceleration method for time sequence database |
| CN115185723A (en) * | 2022-06-02 | 2022-10-14 | 广州市玄武无线科技股份有限公司 | Method, system, device and storage medium for repairing Region meta-information |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110019199A (en) | Data storage, querying method, device, equipment, computer readable storage medium | |
| US10585915B2 (en) | Database sharding | |
| US9454558B2 (en) | Managing an index of a table of a database | |
| US9875272B1 (en) | Method and system for designing a database system for high event rate, while maintaining predictable query performance | |
| CN110008257A (en) | Data processing method, device, system, computer equipment and storage medium | |
| WO2017016423A1 (en) | Real-time new data update method and device | |
| CN109787908A (en) | Server current-limiting method, system, computer equipment and storage medium | |
| US10255307B2 (en) | Database object management for a shared pool of configurable computing resources | |
| JP6242930B2 (en) | Sensor data management device, sensor data management method and program | |
| AU2014201593A1 (en) | Shared cache used to provide zero copy memory mapped database | |
| CN108304484A (en) | Key word matching method and device, electronic equipment and readable storage medium storing program for executing | |
| CN108399175B (en) | Data storage and query method and device | |
| US11537569B2 (en) | Merges using key range data structures | |
| US11321318B2 (en) | Dynamic access paths | |
| CA3009817A1 (en) | Systems and methods for caching task execution | |
| CN107967279A (en) | The data-updating method and device of distributed data base | |
| CN112925783A (en) | Service data processing method and device, electronic equipment and storage medium | |
| CN109033365A (en) | A kind of data processing method and relevant device | |
| Malensek et al. | Autonomous cloud federation for high-throughput queries over voluminous datasets | |
| CN112199426A (en) | Interface call management method, device, server and medium under micro-service architecture | |
| CN111008198A (en) | Service data acquisition method and device, storage medium and electronic equipment | |
| US10552059B2 (en) | Data migration with placement based on access patterns | |
| Vobugari et al. | A roadmap on improved performance-centric cloud storage estimation approach for database system deployment in cloud environment | |
| JP7105718B2 (en) | Information processing device, information processing method, and program | |
| US9342511B2 (en) | Fast selection in hardware or software |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190716 |