CN103488717A - Lock-free data gathering method and lock-free data gathering device - Google Patents
Lock-free data gathering method and lock-free data gathering device Download PDFInfo
- Publication number
- CN103488717A CN103488717A CN201310413005.2A CN201310413005A CN103488717A CN 103488717 A CN103488717 A CN 103488717A CN 201310413005 A CN201310413005 A CN 201310413005A CN 103488717 A CN103488717 A CN 103488717A
- Authority
- CN
- China
- Prior art keywords
- buffer memory
- memory array
- pointer
- packet
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention discloses a lock-free data gathering method and a lock-free data gathering device. The lock-free data gathering method includes: opening up at least one cache memory array in a cache, wherein attributes of each cache memory array include pointer extraction, pointer storage and cache marks used for marking whether pointers stored are ahead of the pointers extracted; establishing at least one storage thread used for storing data packages extracted from client sides in the cache memory arrays and at least one extraction thread used for extracting the data packages from the cache memory arrays so as to enable each cache memory array to be provided with a corresponding storage thread and a corresponding extraction thread; allowing the storage threads and the extraction threads corresponding to the cache memory arrays opened up to perform thread running or thread waiting according to the attributes of the corresponding cache memory arrays so as to extract the data packages in concurrence from at least one client side for convergence. By the lock-free data gather method and device, data convergence of multithreads by concurrently caching can be realized, and efficiency of the data convergence can be improved.
Description
Technical field
The present invention relates to the Computer Applied Technology field, be specifically related to technical field of data storage, relate in particular to without lock convergence method and device.
Background technology
The convergence technology is processed many piece of data or information, be combined into the process of the data that more effectively, more meet user's request, when carrying out convergence, application program after receiving data in real time from network, must carry out buffer memory to the data that receive, other application program is taken out data from buffer memory, and taken out data are converged.
At present, by data deposit buffer memory in, the access of postponing go out data be " atom " operation, a side is when being operated, the opposing party is not operated buffer memory, under conventional situation, because data traffic is very little, generally take the mode that buffer memory " is locked " to guarantee the atomicity operated.But, when data traffic rises to 10Mpbs when above, the speed of the mode that locks deal with data can not meet the requirement of application far away.
Summary of the invention
In view of this, it is a kind of without lock convergence method and device that the embodiment of the present invention provides, and solves the technical matters that above background technology is partly mentioned.
The embodiment of the present invention is by the following technical solutions:
First aspect, it is a kind of without lock convergence method that the embodiment of the present invention provides, and comprising:
Open up at least one buffer memory memory array in buffer memory, the attribute of described buffer memory memory array comprises fetch pointer, deposit pointer and for identifying the described pointer of depositing whether at the buffer memory sign of described fetch pointer front;
Creating at least one is saved in the buffer memory memory array for the packet that will extract from client and deposits thread and at least one for take out the line taking journey of packet from the buffer memory memory array, so that each buffer memory memory array of opening up all has the line taking journey of depositing thread and a correspondence of a correspondence;
Make that each opened up buffer memory memory array is corresponding deposits thread and the line taking journey is carried out thread operation or thread waits according to the attribute of corresponding buffer memory memory array, to walk abreast, from least one client, extract packet and converged.
Second aspect, it is a kind of without lock convergence device that the embodiment of the present invention also provides, and comprising:
Buffer memory memory array creating unit, for open up at least one buffer memory memory array at buffer memory, the attribute of described buffer memory memory array comprises fetch pointer, deposit pointer and for identifying the described pointer of depositing whether at the buffer memory sign of described fetch pointer front;
The thread creation unit, be saved in the buffer memory memory array for the packet that will extract from client and deposit thread and at least one for the line taking journey from buffer memory memory array taking-up packet, so that each buffer memory memory array of opening up all has the line taking journey of depositing thread and a correspondence of a correspondence for creating at least one;
The thread running unit, for making, each opened up buffer memory memory array is corresponding deposits thread and the line taking journey is carried out thread operation or thread waits according to the attribute of corresponding buffer memory memory array, to walk abreast, from least one client, extracts packet and is converged.
The useful technique effect of technology master's case that the embodiment of the present invention proposes is:
The embodiment of the present invention by opening up at least one buffer memory memory array in buffer memory, creating at least one is saved in the buffer memory memory array for the packet that will extract from client and deposits thread and at least one for take out the line taking journey of packet from the buffer memory memory array, so that each buffer memory memory array of opening up all has the line taking journey of depositing thread and a correspondence of a correspondence, make that each opened up buffer memory memory array is corresponding deposits thread and the line taking journey is carried out thread operation or thread waits according to the attribute of corresponding buffer memory memory array, extract packet and converged from least one client to walk abreast.The embodiment of the present invention has been avoided buffer memory " is locked " and the extraction efficiency of restricting data bag can realize that a plurality of threads are concomitantly from carry out convergence by buffer memory, can improve the efficiency of convergence.
The accompanying drawing explanation
In order to be illustrated more clearly in the technical scheme in the embodiment of the present invention, in below describing the embodiment of the present invention, the accompanying drawing of required use is briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, the accompanying drawing that can also obtain other according to content and these accompanying drawings of the embodiment of the present invention.
Fig. 1 is that the specific embodiment of the invention one is described without lock convergence method flow diagram;
Fig. 2 is that the specific embodiment of the invention one is described without lock convergence schematic diagram;
Fig. 3 is the described method flow diagram of depositing thread of the specific embodiment of the invention two;
Fig. 4 is the method flow diagram of the described line taking journey of the specific embodiment of the invention two;
Fig. 5 is that the specific embodiment of the invention two is described without lock convergence schematic diagram;
Fig. 6 is the described structured flowchart without lock convergence device of the specific embodiment of the invention three;
Fig. 7 is the described structured flowchart without lock convergence device of the specific embodiment of the invention four.
Embodiment
For the technical scheme of technical matters that the present invention is solved, employing and the technique effect that reaches clearer, technical scheme below in conjunction with accompanying drawing to the embodiment of the present invention is described in further detail, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.Embodiment based in the present invention, those skilled in the art, not making under the creative work prerequisite the every other embodiment obtained, belong to the scope of protection of the invention.
Further illustrate technical scheme of the present invention below in conjunction with accompanying drawing and by embodiment.
Embodiment mono-
Fig. 1 is that the present embodiment is described without lock convergence method flow diagram, and as shown in Figure 1, the present embodiment is described to be comprised without lock convergence method:
S101, open up at least one buffer memory memory array in buffer memory.
Server is opened up at least one buffer memory memory array in buffer memory, and the attribute of the buffer memory memory array created comprises fetch pointer, deposit pointer and for identifying the described pointer of depositing whether at the buffer memory sign of described fetch pointer front.
In order to make, the buffer memory memory array is corresponding deposits the attribute that thread and line taking journey can be shared this buffer memory memory array, what guarantee this buffer memory memory array deposits thread and line taking journey in the fetch pointer of the corresponding buffer memory memory array of operation, the consistance while depositing pointer and buffer memory sign, can be by the fetch pointer of each buffer memory memory array, deposit pointer and the buffer memory sign can be set to the variable storage of volatile type in the register of server.
S102, create at least one and be saved in the buffer memory memory array for the packet that will extract from client and deposit thread and at least one for take out the line taking journey of packet from the buffer memory memory array.
Server creates at least one and is saved in the buffer memory memory array for the packet that will extract from client and deposits thread and at least one for take out the line taking journey of packet from the buffer memory memory array, so that each buffer memory memory array of opening up all meets: have and only have the thread of depositing of a correspondence, have and only have the line taking journey of a correspondence.
Deposit thread for to corresponding buffer memory memory array, storing packet, the size according to the attribute of corresponding buffer memory array and packet to be stored, determine whether carry out thread waits, after having stored, revises the attribute of corresponding buffer memory memory array.
The line taking journey, for from corresponding buffer memory memory array, extracting packet, after having extracted, is revised the attribute of corresponding buffer memory memory array.
S103, make that each opened up buffer memory memory array is corresponding deposits thread and the line taking journey is carried out thread operation or thread waits according to the attribute of corresponding buffer memory memory array, to walk abreast, from least one client, extract packet and converged.
Server, after step S102 has created thread, can move created all threads, so that each buffer memory memory array is corresponding, deposits thread and the line taking journey is carried out storage and the extraction of data.
For example, as shown in Figure 2, server end has created buffer memory memory array 1, buffer memory memory array 2, buffer memory memory array 3 and buffer memory memory array M etc., being respectively buffer memory memory array 1, buffer memory memory array 2, buffer memory memory array 3 and buffer memory memory array M has created and corresponding deposits thread 1, deposits thread 2, deposits thread 3, deposits thread M, for buffer memory memory array 1, buffer memory memory array 2 have created corresponding line taking journey 1, for buffer memory memory array 3 and buffer memory memory array M have created corresponding line taking journey 2.
Deposit thread 1, deposit thread 2, deposit thread 3, deposit thread M, line taking journey 1 and 2 parallel runnings of line taking journey, the packet that will extract from client 1, client 2, client 3 and client M converge.For example, deposit thread 1 and extract packet from client 1, carry out thread waits according to the determined property of the size of extracted packet and buffer memory memory array 1 and still carry out the packet storage, if carry out the packet storage, after having stored, the attribute of buffer memory memory array 1 is modified.Line taking journey 1 is for from buffer memory memory array 1, mentioning packet, and packet of every extraction, just once revise the attribute of buffer memory memory array 1.
The described technical scheme of the present embodiment, got rid of the mode that buffer memory is locked of the prior art, proposed a kind of without lock convergence method, by opening up the buffer memory memory array, thread and line taking journey are deposited in establishment, so that each buffer memory memory array of opening up all has the line taking journey of depositing thread and a correspondence of a correspondence, make each buffer memory memory array corresponding deposit thread and line taking journey can be concomitantly to access data in the buffer memory memory array, thereby realize that a plurality of threads, concomitantly from by buffer memory, carrying out convergence, can improve the efficiency of convergence.
Embodiment bis-
With embodiment mono-, compare, the present embodiment has been introduced the buffer address queue, and specifically to depositing thread and line taking journey, according to corresponding buffer memory memory array, deposits that pointer, fetch pointer and buffer memory sign carry out the storage of packet and the method for extraction is further elaborated.
What the present embodiment proposed comprises without lock convergence method:
The first step: server is opened up at least one buffer memory memory array in buffer memory, and the attribute of the buffer memory memory array created comprises fetch pointer, deposit pointer and for identifying the described pointer of depositing whether at the buffer memory sign of described fetch pointer front.
Second step: server creates at least one and is saved in the buffer memory memory array for the packet that will extract from client and deposits thread and at least one for take out the line taking journey of packet from the buffer memory memory array, so that each buffer memory memory array of opening up all meets: have and only have the thread of depositing of a correspondence, have and only have the line taking journey of a correspondence.
The 3rd step: server is respectively each line taking journey and opens up at least one corresponding buffer address queue in buffer memory, the address object of the packet that each buffer address queue of opening up is preserved for the buffer memory memory array of storing at least one and opening up, wherein said address object comprises the buffer memory memory array sign for the buffer memory memory array at identification data packet place.
Described address object also comprises start address and the data length of stored packet in buffer memory further.
Perhaps, described address object also comprises relative address and the data length of the start address of stored packet in corresponding buffer memory memory array.
The 4th step: server makes that each opened up buffer memory memory array is corresponding deposits thread and the line taking journey is carried out thread operation or thread waits according to the attribute of corresponding buffer memory memory array, to walk abreast, from least one client, extracts packet and is converged.
Further, in the 4th step, server makes that each opened up buffer memory memory array is corresponding deposits the method that thread and line taking journey carry out thread operation or thread waits according to the attribute of corresponding buffer memory memory array and specifically comprise:
Server make the buffer memory memory array corresponding deposit thread for: receive the packet that at least one client sends, according to the attribute of corresponding buffer memory memory array, received packet is stored to corresponding buffer memory memory array, the address information calculated address object of stored packet is stored in the corresponding buffer address queue of the corresponding line taking journey of corresponding buffer memory memory array;
Server make the corresponding line taking journey of buffer memory memory array for: extract address object from corresponding buffer address queue, obtain the attribute that buffer memory memory array in extracted address object identifies corresponding buffer memory memory array, according to obtained attribute and the address object that extracts, extract packet.
Particularly, Fig. 3 is the described method flow diagram of depositing thread of the present embodiment, and as shown in Figure 3, in the present embodiment, the method for depositing thread comprises:
S301, obtain the size of packet to be stored.
S302, obtain this fetch pointer of depositing the corresponding buffer memory memory array of thread, deposit pointer and buffer memory sign.
S303, judge that whether the buffer memory sign is true, if perform step S304, otherwise execution step S307.
S304, judge fetch pointer and deposit pointer and whether equate, if perform step S305, otherwise execution step S306.
S305, make this deposit thread waits.
S306, judgement are deposited pointer and whether are less than the size of packet to be stored to the space between fetch pointer, if perform step S305, otherwise execution step S310.
Whether the space that S307, judgement are deposited between the last address of pointer and buffer memory memory array is less than the size of packet to be stored, if perform step S308, otherwise performs step S310.
S308, to judge whether this start address of depositing the corresponding buffer memory memory array of thread is less than to the space between fetch pointer to be stored, if perform step S305, otherwise execution step S309.
The purpose of this step is: if described fetch pointer to the insufficient space at the end of described buffer memory memory array to store packet to be stored, skip this space, packet to be stored is stored into to the start address of buffer memory memory array.
In another embodiment of the present invention, this step also can be: judge whether the remaining space of described buffer memory memory array is less than the size of packet to be stored, if perform step S305, otherwise execution step S309.
S309, the buffer memory sign is changed to very to execution step S310.
S310, packet to be stored is stored, finished.
The wherein said step that packet to be stored is stored specifically comprises: according to corresponding buffer memory memory array deposit pointer by packet to be stored store in corresponding buffer memory memory array and the corresponding buffer memory memory array of change institute deposit pointer, the address by packet to be stored in corresponding buffer memory memory array be recorded in the corresponding buffer address queue of the corresponding line taking journey of corresponding buffer memory memory array.
Finally, the packet described at least two line taking journeys extracted is converged.
Particularly, Fig. 4 is the method flow diagram of the described line taking journey of the present embodiment, and as shown in Figure 4, in the present embodiment, the method for line taking journey comprises:
S401, from corresponding buffer address queue, extract address object.
S402, according to extracted address object, to the buffer memory memory array in described address object, identify in corresponding buffer memory memory array and extract packet.
It should be noted that, if the address object in the present embodiment comprises the buffer memory memory array sign of buffer memory memory array, start address and the data length of packet in buffer memory of storing, the directly start address in buffer memory and the direct reading out data of data length according to described packet; If the address object in the present embodiment comprises relative address and the data length of the start address of packet in corresponding buffer memory memory array that the buffer memory memory array of buffer memory memory array identifies, stores, at first by described buffer memory memory array sign, find corresponding buffer memory memory array, from the fetch pointer of described buffer memory memory array, the data that the taking-up data length is the data length in described data object.
S403, according to described address object, change the fetch pointer in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array.
S404, compare and whether diminish before judging described fetch pointer and change, if perform step S405, otherwise execution step S406.
S405, described buffer memory sign are set to no.
S406, do not change described buffer memory sign.
For example, as shown in Figure 5, server end has created buffer memory memory array 1, buffer memory memory array 2, buffer memory memory array 3 and buffer memory memory array M etc.; Being respectively buffer memory memory array 1, buffer memory memory array 2, buffer memory memory array 3 and buffer memory memory array M has created and corresponding deposits thread 1, deposits thread 2, deposits thread 3, deposits thread M; For buffer memory memory array 1, buffer memory memory array 2 have created corresponding line taking journey 1, for buffer memory memory array 3 and buffer memory memory array M have created corresponding line taking journey 2; For line taking journey 1 has created buffer address queue 1, for line taking journey 2 has created buffer address queue 2.
Deposit thread 1, deposit thread 2, deposit thread 3, deposit thread M, line taking journey 1 and 2 parallel runnings of line taking journey, the packet that will extract from client 1, client 2, client 3 and client M converge.For example, deposit thread 1 and extract packet from client 1, carry out thread waits according to the determined property of the size of extracted packet and buffer memory memory array 1 and still carry out the packet storage, if carry out the packet storage, after having stored, attribute to buffer memory memory array 1 is modified, and the address information calculated address object of stored packet is stored in address queue 1.Line taking journey 1 is for extracting address object from buffer memory address queue 1, the packet that this address object points to is stored in buffer memory memory array 1, obtain the attribute of buffer memory memory array 1, according to the attribute of buffer memory memory array 1 and the address object extracted, extract packet.
With embodiment mono-, compare, the present embodiment has been introduced the buffer address queue, be respectively each line taking journey and open up at least one corresponding buffer address queue in buffer memory, the address object of the packet that each buffer address queue of opening up is preserved for the buffer memory memory array of storing at least one and opening up, respectively deposit thread and line taking journey respectively according to the fetch pointer of corresponding buffer memory memory array, deposit pointer and buffer memory sign and control the thread progress and guarantee efficiently access data concomitantly, can improve the efficiency of convergence.
Embodiment tri-
Fig. 6 is the described structured flowchart without lock convergence device of the present embodiment, and as shown in Figure 6, the present embodiment is described to be comprised without lock convergence device:
Buffer memory memory array creating unit 601, for open up at least one buffer memory memory array at buffer memory, the attribute of described buffer memory memory array comprises fetch pointer, deposit pointer and for identifying the described pointer of depositing whether at the buffer memory sign of described fetch pointer front.
Server is opened up at least one buffer memory memory array in buffer memory, and the attribute of the buffer memory memory array created comprises fetch pointer, deposit pointer and for identifying the described pointer of depositing whether at the buffer memory sign of described fetch pointer front.
In order to make, the buffer memory memory array is corresponding deposits the attribute that thread and line taking journey can be shared this buffer memory memory array, what guarantee this buffer memory memory array deposits thread and line taking journey in the fetch pointer of the corresponding buffer memory memory array of operation, the consistance while depositing pointer and buffer memory sign, can be by the fetch pointer of each buffer memory memory array, deposit pointer and the buffer memory sign can be set to the variable storage of volatile type in the register of server.
Server creates at least one and is saved in the buffer memory memory array for the packet that will extract from client and deposits thread and at least one for take out the line taking journey of packet from the buffer memory memory array, so that each buffer memory memory array of opening up all meets: have and only have the thread of depositing of a correspondence, have and only have the line taking journey of a correspondence.
Deposit thread for to corresponding buffer memory memory array, storing packet, the size according to the attribute of corresponding buffer memory array and packet to be stored, determine whether carry out thread waits, after having stored, revises the attribute of corresponding buffer memory memory array.
The line taking journey, for from corresponding buffer memory memory array, extracting packet, after having extracted, is revised the attribute of corresponding buffer memory memory array.
The described technical scheme of the present embodiment, got rid of the mode that buffer memory is locked of the prior art, proposed a kind of without lock convergence method, by opening up the buffer memory memory array, thread and line taking journey are deposited in establishment, so that each buffer memory memory array of opening up all has the line taking journey of depositing thread and a correspondence of a correspondence, make each buffer memory memory array corresponding deposit thread and line taking journey can be concomitantly to access data in the buffer memory memory array, thereby realize that a plurality of threads, concomitantly from by buffer memory, carrying out convergence, can improve the efficiency of convergence.
Embodiment tetra-
Fig. 7 is the described structured flowchart without lock convergence device of the present embodiment, and as shown in Figure 7, the present embodiment is described to be comprised without lock convergence device:
Buffer memory memory array creating unit 701, for open up at least one buffer memory memory array at buffer memory, the attribute of described buffer memory memory array comprises fetch pointer, deposit pointer and for identifying the described pointer of depositing whether at the buffer memory sign of described fetch pointer front.
Address queue's creating unit 703, be saved in for the packet that will extract from client for create at least one in thread creating unit 702 that the buffer memory memory array is deposited thread and at least one is for after the line taking journey from buffer memory memory array taking-up packet, be respectively each line taking journey and open up at least one corresponding buffer address queue in buffer memory, the address object of the packet that each buffer address queue of opening up is preserved for the buffer memory memory array of storing at least one and opening up, wherein said address object comprises the buffer memory memory array sign for the buffer memory memory array at identification data packet place.
Further, thread running unit 704 comprises that depositing thread moves subelement 7041 and line taking journey operation subelement 7042.
Deposit thread operation subelement 7041 for make the buffer memory memory array corresponding deposit thread for: receive the packet that at least one client sends, according to the attribute of corresponding buffer memory memory array, received packet is stored to corresponding buffer memory memory array, the address information calculated address object of stored packet is stored in the corresponding buffer address queue of the corresponding line taking journey of corresponding buffer memory memory array.
Line taking journey operation subelement 7042 for make the corresponding line taking journey of buffer memory memory array for: extract address object from corresponding buffer address queue, obtain the attribute that buffer memory memory array in extracted address object identifies corresponding buffer memory memory array, according to obtained attribute and the address object that extracts, extract packet.
Further, described depositing described in thread operation subelement 7041 stored received packet to comprise to corresponding buffer memory memory array according to the attribute of corresponding buffer memory memory array:
Obtain the size of received packet, obtain corresponding buffer memory memory array fetch pointer, deposit pointer and buffer memory sign;
If described buffer memory is masked as true and described fetch pointer and described to deposit pointer unequal, judge whether the described pointer of depositing is less than the size of received packet to the space between described fetch pointer, if make the described thread waits of depositing, otherwise received packet stored;
Equate with the described pointer of depositing if described buffer memory is masked as true and described fetch pointer, make the described thread waits of depositing;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is less than the size of received packet, judge whether the remaining space of corresponding buffer memory memory array is less than the size of described packet to be stored, if make the described thread waits of depositing, otherwise described buffer memory sign is changed to very, received packet is stored;
In another embodiment of the present invention, above-mentioned steps also can be: if described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is less than the size of received packet, whether the start address that judges described buffer memory memory array is less than the size of described packet to be stored to the space between described fetch pointer, if make the described thread waits of depositing, otherwise described buffer memory sign is changed to very, received packet is stored;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is not less than the size of described packet to be stored, received packet stored;
The wherein said step that received packet is stored comprises: store in corresponding buffer memory memory array by received packet and change the described pointer of depositing according to the described pointer of depositing, the address object by received packet in corresponding buffer memory memory array is recorded in the specified buffer address queue of corresponding buffer memory memory array.
Further, extracting packet according to obtained attribute and the address object extracted described in described line taking journey operation subelement 7042 specifically comprises:
Identify in corresponding buffer memory memory array and extract packet to the buffer memory memory array in described address object according to extracted address object;
Change the fetch pointer in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array according to described address object;
Judge that described address object changes fetch pointer in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array and whether diminish with comparing before change, if the buffer memory sign that described address object is changed in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array is set to no.
Further, described address object also comprises start address and the data length of stored packet in buffer memory.
Further, described address object also comprises relative address and the data length of the start address of stored packet in corresponding buffer memory memory array.
With embodiment tri-, compare, the present embodiment has been introduced address queue's creating unit 703, open up at least one corresponding buffer address queue for being respectively each line taking journey at buffer memory, the address object of the packet that each buffer address queue of opening up is preserved for the buffer memory memory array of storing at least one and opening up, make respectively deposit thread and line taking journey respectively according to the fetch pointer of corresponding buffer memory memory array, deposit pointer and buffer memory sign and control the thread progress and guarantee efficiently access data concomitantly, can improve the efficiency of convergence.
Below that data under the described technical scheme of the embodiment of the present invention are taken out to the situation that efficiency and the traditional mode that buffer memory " is locked " come the experimental result of access data to compare:
Network environment: LAN (Local Area Network) gigabit bandwidth.
Transmission mode: adopt ICP/IP protocol to carry out data transmission.
Result: at packet, during in the 200B left and right, data are taken out speed at 25-35Mpbs; When data, between 1KB to 4KB the time, data are taken out speed only in the 70-80Mpbs left and right; When data, when 1M is above, data are taken out speed only more than 80Mpbs.
And adopt when the mode that in buffer memory, each buffer memory memory array locks is tested, under different pieces of information bag size cases, data are taken out speed only in the 10Mpbs left and right, because not obtaining the thread of lock can only wait for, and thread carries out executable state by waiting status and can consume ample resources, cause whole access rate can't meet the large-scale data transmission at all.
All or part of content in the technical scheme that above embodiment provides can realize by software programming, and its software program is stored in the storage medium can read, storage medium for example: the hard disk in computing machine, CD or floppy disk.
Note, above are only preferred embodiment of the present invention and institute's application technology principle.Skilled person in the art will appreciate that and the invention is not restricted to specific embodiment described here, can carry out for a person skilled in the art various obvious variations, readjust and substitute and can not break away from protection scope of the present invention.Therefore, although by above embodiment, the present invention is described in further detail, the present invention is not limited only to above embodiment, in the situation that do not break away from the present invention's design, can also comprise more other equivalent embodiment, and scope of the present invention is determined by appended claim scope.
Claims (16)
1. a nothing lock convergence method, is characterized in that, comprising:
Open up at least one buffer memory memory array in buffer memory, the attribute of described buffer memory memory array comprises fetch pointer, deposit pointer and for identifying the described pointer of depositing whether at the buffer memory sign of described fetch pointer front;
Creating at least one is saved in the buffer memory memory array for the packet that will extract from client and deposits thread and at least one for take out the line taking journey of packet from the buffer memory memory array, so that each buffer memory memory array of opening up all has the line taking journey of depositing thread and a correspondence of a correspondence;
Make that each opened up buffer memory memory array is corresponding deposits thread and the line taking journey is carried out thread operation or thread waits according to the attribute of corresponding buffer memory memory array, to walk abreast, from least one client, extract packet and converged.
2. as claimed in claim 1 without lock convergence method, it is characterized in that, create at least one and be saved in the buffer memory memory array for the packet that will extract from client and also comprise after depositing thread and at least one step for the line taking journey of taking out packet from the buffer memory memory array:
Be respectively each line taking journey and open up at least one corresponding buffer address queue in buffer memory, the address object of the packet that each buffer address queue of opening up is preserved for the buffer memory memory array of storing at least one and opening up, wherein said address object comprises the buffer memory memory array sign for the buffer memory memory array at identification data packet place.
3. as claimed in claim 2ly without lock convergence method, it is characterized in that, make that each opened up buffer memory memory array is corresponding deposits the step that thread and line taking journey carry out thread operation or thread waits according to the attribute of corresponding buffer memory memory array and comprise:
Make the buffer memory memory array corresponding deposit thread for: receive the packet that at least one client sends, according to the attribute of corresponding buffer memory memory array, received packet is stored to corresponding buffer memory memory array, the address information calculated address object of stored packet is stored in the corresponding buffer address queue of the corresponding line taking journey of corresponding buffer memory memory array;
Make the corresponding line taking journey of buffer memory memory array for: extract address object from corresponding buffer address queue, obtain the attribute that buffer memory memory array in extracted address object identifies corresponding buffer memory memory array, according to obtained attribute and the address object that extracts, extract packet.
4. nothing lock convergence method as claimed in claim 3, is characterized in that, the described attribute according to corresponding buffer memory memory array is stored step from received packet to corresponding buffer memory memory array comprises:
Obtain the size of received packet, obtain corresponding buffer memory memory array fetch pointer, deposit pointer and buffer memory sign;
If described buffer memory is masked as true and described fetch pointer and described to deposit pointer unequal, judge whether the described pointer of depositing is less than the size of received packet to the space between described fetch pointer, if make the described thread waits of depositing, otherwise received packet stored;
Equate with the described pointer of depositing if described buffer memory is masked as true and described fetch pointer, make the described thread waits of depositing;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is less than the size of received packet, judge whether the remaining space of corresponding buffer memory memory array is less than the size of described packet to be stored, if make the described thread waits of depositing, otherwise described buffer memory sign is changed to very, received packet is stored;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is not less than the size of described packet to be stored, received packet stored;
The wherein said step that received packet is stored comprises: store in corresponding buffer memory memory array by received packet and change the described pointer of depositing according to the described pointer of depositing, the address object by received packet in corresponding buffer memory memory array is recorded in the specified buffer address queue of corresponding buffer memory memory array.
5. nothing lock convergence method as claimed in claim 3, is characterized in that, the described attribute according to corresponding buffer memory memory array is stored step from received packet to corresponding buffer memory memory array comprises:
Obtain the size of received packet, obtain corresponding buffer memory memory array fetch pointer, deposit pointer and buffer memory sign;
If described buffer memory is masked as true and described fetch pointer and described to deposit pointer unequal, judge whether the described pointer of depositing is less than the size of received packet to the space between described fetch pointer, if make the described thread waits of depositing, otherwise received packet stored;
Equate with the described pointer of depositing if described buffer memory is masked as true and described fetch pointer, make the described thread waits of depositing;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is less than the size of received packet, whether the start address that judges described buffer memory memory array is less than the size of described packet to be stored to the space between described fetch pointer, if make the described thread waits of depositing, otherwise described buffer memory sign is changed to very, received packet is stored;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is not less than the size of described packet to be stored, received packet stored;
The wherein said step that received packet is stored comprises: store in corresponding buffer memory memory array by received packet and change the described pointer of depositing according to the described pointer of depositing, the address object by received packet in corresponding buffer memory memory array is recorded in the specified buffer address queue of corresponding buffer memory memory array.
6. nothing lock convergence method as claimed in claim 3, is characterized in that, according to obtained attribute and the step that the address object extracted extracts packet, specifically comprises:
Identify in corresponding buffer memory memory array and extract packet to the buffer memory memory array in described address object according to extracted address object;
Change the fetch pointer in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array according to described address object;
Judge that described address object changes fetch pointer in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array and whether diminish with comparing before change, if the buffer memory sign that described address object is changed in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array is set to no.
7. nothing lock convergence method as claimed in claim 2, is characterized in that, described address object also comprises start address and the data length of stored packet in buffer memory.
8. nothing lock convergence method as claimed in claim 2, is characterized in that, described address object also comprises relative address and the data length of the start address of stored packet in corresponding buffer memory memory array.
9. a nothing lock convergence device, is characterized in that, comprising:
Buffer memory memory array creating unit, for open up at least one buffer memory memory array at buffer memory, the attribute of described buffer memory memory array comprises fetch pointer, deposit pointer and for identifying the described pointer of depositing whether at the buffer memory sign of described fetch pointer front;
The thread creation unit, be saved in the buffer memory memory array for the packet that will extract from client and deposit thread and at least one for the line taking journey from buffer memory memory array taking-up packet, so that each buffer memory memory array of opening up all has the line taking journey of depositing thread and a correspondence of a correspondence for creating at least one;
The thread running unit, for making, each opened up buffer memory memory array is corresponding deposits thread and the line taking journey is carried out thread operation or thread waits according to the attribute of corresponding buffer memory memory array, to walk abreast, from least one client, extracts packet and is converged.
10. as claimed in claim 9 without lock convergence device, it is characterized in that, also comprise address queue's creating unit, be saved in the buffer memory memory array for the packet that will extract from client and deposit thread and at least one for after the line taking journey from buffer memory memory array taking-up packet for create at least one in the thread creating unit, be respectively each line taking journey and open up at least one corresponding buffer address queue in buffer memory, the address object of the packet that each buffer address queue of opening up is preserved for the buffer memory memory array of storing at least one and opening up, wherein said address object comprises the buffer memory memory array sign for the buffer memory memory array at identification data packet place.
11. as claimed in claim 10, without lock convergence device, it is characterized in that, described thread running unit comprises deposits thread operation subelement and line taking journey operation subelement,
Deposit thread operation subelement for make the buffer memory memory array corresponding deposit thread for: receive the packet that at least one client sends, according to the attribute of corresponding buffer memory memory array, received packet is stored to corresponding buffer memory memory array, the address information calculated address object of stored packet is stored in the corresponding buffer address queue of the corresponding line taking journey of corresponding buffer memory memory array;
Line taking journey operation subelement for make the corresponding line taking journey of buffer memory memory array for: extract address object from corresponding buffer address queue, obtain the attribute that buffer memory memory array in extracted address object identifies corresponding buffer memory memory array, according to obtained attribute and the address object that extracts, extract packet.
12. as claimed in claim 11, without lock convergence device, it is characterized in that, described depositing described in thread operation subelement stored received packet to comprise to corresponding buffer memory memory array according to the attribute of corresponding buffer memory memory array:
Obtain the size of received packet, obtain corresponding buffer memory memory array fetch pointer, deposit pointer and buffer memory sign;
If described buffer memory is masked as true and described fetch pointer and described to deposit pointer unequal, judge whether the described pointer of depositing is less than the size of received packet to the space between described fetch pointer, if make the described thread waits of depositing, otherwise received packet stored;
Equate with the described pointer of depositing if described buffer memory is masked as true and described fetch pointer, make the described thread waits of depositing;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is less than the size of received packet, judge whether the remaining space of corresponding buffer memory memory array is less than the size of described packet to be stored, if make the described thread waits of depositing, otherwise described buffer memory sign is changed to very, received packet is stored;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is not less than the size of described packet to be stored, received packet stored;
The wherein said step that received packet is stored comprises: store in corresponding buffer memory memory array by received packet and change the described pointer of depositing according to the described pointer of depositing, the address object by received packet in corresponding buffer memory memory array is recorded in the specified buffer address queue of corresponding buffer memory memory array.
13. as claimed in claim 11, without lock convergence device, it is characterized in that, described depositing described in thread operation subelement stored received packet to comprise to corresponding buffer memory memory array according to the attribute of corresponding buffer memory memory array:
Obtain the size of received packet, obtain corresponding buffer memory memory array fetch pointer, deposit pointer and buffer memory sign;
If described buffer memory is masked as true and described fetch pointer and described to deposit pointer unequal, judge whether the described pointer of depositing is less than the size of received packet to the space between described fetch pointer, if make the described thread waits of depositing, otherwise received packet stored;
Equate with the described pointer of depositing if described buffer memory is masked as true and described fetch pointer, make the described thread waits of depositing;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is less than the size of received packet, whether the start address that judges described buffer memory memory array is less than the size of described packet to be stored to the space between described fetch pointer, if make the described thread waits of depositing, otherwise described buffer memory sign is changed to very, received packet is stored;
If described buffer memory is masked as vacation, and the space between described last address of depositing pointer and described buffer memory memory array is not less than the size of described packet to be stored, received packet stored;
The wherein said step that received packet is stored comprises: store in corresponding buffer memory memory array by received packet and change the described pointer of depositing according to the described pointer of depositing, the address object by received packet in corresponding buffer memory memory array is recorded in the specified buffer address queue of corresponding buffer memory memory array.
It is characterized in that without lock convergence device 14. as claimed in claim 11, extract packet according to obtained attribute and the address object extracted described in described line taking journey operation subelement and specifically comprise:
Identify in corresponding buffer memory memory array and extract packet to the buffer memory memory array in described address object according to extracted address object;
Change the fetch pointer in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array according to described address object;
Judge that described address object changes fetch pointer in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array and whether diminish with comparing before change, if the buffer memory sign that described address object is changed in the attribute that buffer memory memory array in described address object identifies corresponding buffer memory memory array is set to no.
15. as claimed in claim 10, without lock convergence device, it is characterized in that, described address object also comprises start address and the data length of stored packet in buffer memory.
16. as claimed in claim 10, without lock convergence device, it is characterized in that, described address object also comprises relative address and the data length of the start address of stored packet in corresponding buffer memory memory array.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310413005.2A CN103488717B (en) | 2013-09-11 | 2013-09-11 | Lock-free data gathering method and lock-free data gathering device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310413005.2A CN103488717B (en) | 2013-09-11 | 2013-09-11 | Lock-free data gathering method and lock-free data gathering device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103488717A true CN103488717A (en) | 2014-01-01 |
| CN103488717B CN103488717B (en) | 2017-02-22 |
Family
ID=49828943
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310413005.2A Active CN103488717B (en) | 2013-09-11 | 2013-09-11 | Lock-free data gathering method and lock-free data gathering device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103488717B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850507A (en) * | 2014-02-18 | 2015-08-19 | 腾讯科技（深圳）有限公司 | Data caching method and data caching device |
| CN106354572A (en) * | 2016-08-31 | 2017-01-25 | 成都科来软件有限公司 | Multi-thread data transmission method |
| CN106789917A (en) * | 2016-11-25 | 2017-05-31 | 北京百家互联科技有限公司 | Data package processing method and device |
| CN106909321A (en) * | 2017-02-24 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of control method and device based on storage system |
| CN113176896A (en) * | 2021-03-19 | 2021-07-27 | 中盈优创资讯科技有限公司 | Method for randomly taking out object based on single-in single-out lock-free queue |
| CN113778674A (en) * | 2021-08-31 | 2021-12-10 | 上海弘积信息科技有限公司 | Lock-free implementation method of load balancing equipment configuration management under multi-core |
| CN114900713A (en) * | 2022-07-13 | 2022-08-12 | 深圳市必提教育科技有限公司 | Video clip processing method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101127989A (en) * | 2007-09-11 | 2008-02-20 | 中兴通讯股份有限公司 | A method for supporting hypertext transmission stream media service of mobile phone |
| CN101631139A (en) * | 2009-05-19 | 2010-01-20 | 华耀环宇科技（北京）有限公司 | Load balancing software architecture based on multi-core platform and method therefor |
| CN102053923A (en) * | 2009-11-05 | 2011-05-11 | 北京金山软件有限公司 | Storage method and storage device for logbook data |
| US20120120965A1 (en) * | 2010-11-12 | 2012-05-17 | Alcatel-Lucent Usa Inc. | Lock-less and zero copy messaging scheme for telecommunication network applications |
-
2013
- 2013-09-11 CN CN201310413005.2A patent/CN103488717B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101127989A (en) * | 2007-09-11 | 2008-02-20 | 中兴通讯股份有限公司 | A method for supporting hypertext transmission stream media service of mobile phone |
| CN101631139A (en) * | 2009-05-19 | 2010-01-20 | 华耀环宇科技（北京）有限公司 | Load balancing software architecture based on multi-core platform and method therefor |
| CN102053923A (en) * | 2009-11-05 | 2011-05-11 | 北京金山软件有限公司 | Storage method and storage device for logbook data |
| US20120120965A1 (en) * | 2010-11-12 | 2012-05-17 | Alcatel-Lucent Usa Inc. | Lock-less and zero copy messaging scheme for telecommunication network applications |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850507A (en) * | 2014-02-18 | 2015-08-19 | 腾讯科技（深圳）有限公司 | Data caching method and data caching device |
| CN104850507B (en) * | 2014-02-18 | 2019-03-15 | 腾讯科技（深圳）有限公司 | A kind of data cache method and data buffer storage |
| CN106354572A (en) * | 2016-08-31 | 2017-01-25 | 成都科来软件有限公司 | Multi-thread data transmission method |
| CN106789917A (en) * | 2016-11-25 | 2017-05-31 | 北京百家互联科技有限公司 | Data package processing method and device |
| CN106789917B (en) * | 2016-11-25 | 2019-10-01 | 北京百家互联科技有限公司 | Data package processing method and device |
| CN106909321A (en) * | 2017-02-24 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of control method and device based on storage system |
| CN113176896A (en) * | 2021-03-19 | 2021-07-27 | 中盈优创资讯科技有限公司 | Method for randomly taking out object based on single-in single-out lock-free queue |
| CN113778674A (en) * | 2021-08-31 | 2021-12-10 | 上海弘积信息科技有限公司 | Lock-free implementation method of load balancing equipment configuration management under multi-core |
| CN114900713A (en) * | 2022-07-13 | 2022-08-12 | 深圳市必提教育科技有限公司 | Video clip processing method and system |
| CN114900713B (en) * | 2022-07-13 | 2022-09-30 | 深圳市必提教育科技有限公司 | Video clip processing method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103488717B (en) | 2017-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103488717A (en) | Lock-free data gathering method and lock-free data gathering device | |
| CN110113420B (en) | NVM-based distributed message queue management system | |
| CN101667144B (en) | Virtual machine communication method based on shared memory | |
| CN103218176B (en) | Data processing method and device | |
| CN101267361B (en) | A high-speed network data packet capturing method based on zero duplication technology | |
| CN102195874A (en) | Pre-fetching of data packets | |
| CN111182008B (en) | Establishing socket connections in user space | |
| CN105677904B (en) | Small documents storage method and device based on distributed file system | |
| CN102064977A (en) | Graphics processing unit (GPU) based method for detecting message content of high-speed network | |
| CN103927305B (en) | It is a kind of that the method and apparatus being controlled is overflowed to internal memory | |
| CN102420771B (en) | Method for increasing concurrent transmission control protocol (TCP) connection speed in high-speed network environment | |
| CN106570145B (en) | Distributed database result caching method based on hierarchical mapping | |
| CN101841438A (en) | Method or system for accessing and storing stream records of massive concurrent TCP streams | |
| US11385900B2 (en) | Accessing queue data | |
| CN104320448A (en) | Method and device for accelerating caching and prefetching of computing device based on big data | |
| CN104571946B (en) | A kind of storage arrangement and its access method for supporting logic circuit quick search | |
| US20130067160A1 (en) | Producer-consumer data transfer using piecewise circular queue | |
| CN103020003A (en) | Multi-core program determinacy replay-facing memory competition recording device and control method thereof | |
| CN106033428B (en) | The selection method of uniform resource locator and the selection device of uniform resource locator | |
| CN111176855A (en) | Establishing queues between threads in user space | |
| CN109783589A (en) | The method, apparatus and storage medium of electronic map parsing address | |
| CN103414603B (en) | Ipv6 deep packet inspection method based on Hash method for folding | |
| US20110246582A1 (en) | Message Passing with Queues and Channels | |
| CN110020272A (en) | Caching method, device and computer storage medium | |
| CN104391947B (en) | Magnanimity GIS data real-time processing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |