CN101119278A - Method and system for processing mass data - Google Patents
Method and system for processing mass data Download PDFInfo
- Publication number
- CN101119278A CN101119278A CNA2007100302529A CN200710030252A CN101119278A CN 101119278 A CN101119278 A CN 101119278A CN A2007100302529 A CNA2007100302529 A CN A2007100302529A CN 200710030252 A CN200710030252 A CN 200710030252A CN 101119278 A CN101119278 A CN 101119278A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- configuration file
- massive
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a mass data processing method and system. When reading the mass data documents, the scale is the range of the data which are read every time according to the scheduled scale to read data; the data which are read every time are saved and the positioning information of the data which are read every time is recorded; the attribute information of the scheduled springing events is recorded; when the client terminal sending an order to acquire the saved mass data documents, the needed mass data documents is searched based on the marking information in the order and the data in the mass data documents are sent according to the scheduled rules; when the client terminal sending an order to process the acquired mass data documents, if the order is an scheduled trigger event, the attribute information of the relative trigger event is amended and the amended attribute information is transmitted to the client terminal. The present invention can realize the interactive operation of the mass data.
Description
Technical field
The present invention relates to treatment technology to mass data.
Background technology
Along with the development of information technology, network becomes the important channel that information is propagated, and the user is by network access server, to obtain needed data.Transmit under limited bandwidth in order to guarantee data, usually need compress in advance data, when server is received user instruction, complete compressed file is transmitted through the network to client, client is carried out the decompress(ion) demonstration to it after receiving compressed file.
Development along with Digital image technology and multimedia information technology, the application of mass data is more and more general, as aerospace remote sensing image data, common and thematic maps digitalized data, document history information and multimedia integrated data, medical image etc.At present in the processing method of data, client need all download to the compressed file that desire shows in the internal memory, again the compressed file in the internal memory is all opened, because the data volume of mass data is very big, handle massive data files and need sizable memory headroom and transmission bandwidth, therefore, in the prior art for the normally passive reception of mass data user, and can't in receiving course, the video data on the screen be handled, for example a certain frame is processed, operations such as modification, can't realize interactive operation, thereby can not satisfy user's demand, reduce user experience.
Summary of the invention
The invention provides a kind of method and system of handling mass data, be applicable to the transmission massive data files, and can realize user's interactive operation, improve user experience.
The invention provides a kind of method of handling mass data, when reading massive data files, according to predetermined number range reading of data, described number range is the scope of the data that at every turn read; The data that at every turn read are preserved, and write down the locating information of the described data that at every turn read; The attribute information of record preset trigger incident; When client sends instruction and obtains the massive data files of being stored, search needed massive data files according to the identification information in the described instruction, transmit data in the described massive data files according to pre-defined rule; When described client transmission instruction is handled the massive data files that is obtained, if described instruction is the preset trigger incident, then revise the attribute information of relative trigger incident, and amended attribute information is transferred to described client.
The present invention also provides a kind of system that handles mass data, comprises data process subsystem and professional control subsystem;
Wherein, described data process subsystem is used to read and preserves massive data files and locating information; And the attribute information of preset trigger incident preserved in first configuration file;
The massive data files of being preserved comprises at least one data block, and each data block comprises the data that read according to predetermined number range respectively, and described number range is the scope of the data that at every turn read; Described locating information is the locating information of described data block correspondence;
Described professional control subsystem, be used to obtain described massive data files, locating information and described first configuration file, when the massive data files of being stored is obtained in client transmission instruction, search needed massive data files according to the identification information in the described instruction, transmit data in the described massive data files according to pre-defined rule; When described client transmission instruction is handled the massive data files that is obtained, if described instruction is the preset trigger incident, then revise described first configuration file, and amended first configuration file is transferred to described client.
Because possible trigger event when the present invention sets in advance the user and handles massive data files, and the attribute information of record relative trigger incident, be sent to client after when receiving user instruction, corresponding attribute information being handled, thereby the user can be handled the massive data files of current demonstration, realized the interactivity of operation; In addition, the present invention preserves the massive data files piecemeal, according to predetermined rule the partial data in each data block or the data block is transmitted during transmission at every turn, and also non-once all transmits all data that the user needs, because each data quantity transmitted is less, therefore need amplify, dwindle etc. when handling current data presented as the user, can be with needed transfer of data to the user.
Further, owing to when access data, massive data files is divided into a plurality of data blocks stores, make to write with reading of data and can in a plurality of data blocks, carry out simultaneously, thereby improved the access speed of data and the transmission performance of data.
Description of drawings
Fig. 1 handles the flow chart of the method for mass data for one embodiment of the invention;
Fig. 2 handles the structured flowchart of the distributed system of mass data for one embodiment of the invention;
Fig. 3 is the workflow diagram of one embodiment of the invention data process subsystem;
Fig. 4 is the workflow diagram of the professional control subsystem of one embodiment of the invention;
Fig. 5 is the workflow diagram of one embodiment of the invention client-end subsystem.
Embodiment
The method of processing mass data disclosed by the invention comprises to the process of the piecemeal of massive data files compression and to mass data to be transmitted and procedure for displaying, below the implementation of each process is set forth.
As shown in Figure 1, because the amount of information of mass data is very big, usually mass data is compressed, so that in transmission over networks, therefore the present invention sets in advance the scope (step 100) of the data that at every turn read, when reading massive data files, according to predetermined number range reading of data (step 101); Preserve (step 102) after the data compression that will at every turn read then, like this, just whole massive data files is divided into several data blocks.Because each data block occupies certain position in massive data files, in order when client needs, to extract the massive data files of being stored, therefore, the locating information of each data block correspondence can be preserved, in order to extract conveniently, can also preserve the information (step 103) such as size of each data block, when proposing corresponding request with convenient user, can massive data files can be restored thus according to the information searching of step 103 record to the corresponding data piece.Massive data files of the present invention can comprise mass image data, video, audio data file etc.
When receiving user instruction, user instruction can carry the identification information of the needed massive data files of user usually, can search needed massive data files (step 104) according to the identification information in the instruction, in addition, can set in advance the rule of transmits data packets according to factors such as transmission bandwidth and user's request and client display properties, massive data files is transferred to client (step 105) according to pre-defined rule; In one embodiment of the invention, the data area of each transmission is consistent with the number range of reading of data, promptly transmits a data block at every turn.Consider transmission bandwidth and factors such as user's request and client display properties, set pre-defined rule can further include: according to the resolution information in the described instruction data of corresponding resolution are transmitted, so that can make the recipient view required data in the short as far as possible stand-by period.Because need be according to the resolution sizes transmission data of client, and for different clients, its resolution may have nothing in common with each other, therefore massive data files is divided into each data block when storing, need be according to the resolution of different clients, the data block that each data block that reads is compressed into different resolution is preserved, and write down corresponding resolution information; When receiving user instruction, find needed massive data files after, according to the resolution information in the described instruction data block of corresponding resolution is transmitted one by one.Above-mentioned with the detailed process hereinafter in detail introduction of data block according to different resolution compression and corresponding demonstration.
When client shows massive data files, obtain the locating information of the data of being transmitted, the data based corresponding locating information of being transmitted is resolved, and in corresponding position display.
In addition, (for example, the user amplified, dwindles, revises, reconfigures etc. the instruction that certain partial data is handled to possible trigger event to the part viewing area in the massive data files of current display area when the present invention also set in advance the user and handles massive data files; Or the data of current display area are marked, increase the instruction that special efficacy, audio frequency etc. do not relate to the mass data of being transmitted), and the attribute information (step 103) of record preset trigger incident; When described client transmission instruction is handled the massive data files that is obtained, if described instruction is the preset trigger incident, then revise the attribute information of relative trigger incident, and amended attribute information is transferred to described client (step 106); Simultaneously, when if this trigger event relates to the processing of data (for example, instruction such as amplify, dwindle), then when user instruction that the data that receive current display area change, change the positional information of data according to resolution information in the user instruction and needs in the viewing area, obtain the data block of corresponding resolution, then these data blocks are transferred to client according to said process.Thus, the system that makes is sending, transmit and receive various mass datas and (comprise literal, figure, sound and image etc.) time realize the real-time interactive operation, make the recipient in receiving course, can process to a certain frame of viewing area, revise, amplify and dwindle, and data presented made amendment and operate (for example open a plurality of windows on a screen and carry out interactive operation), even can on same moving image, be combined into different sound according to different needs, can also get on directly the be added to video image of activity of spectators scene by video camera.
As mentioned above, because the present invention when access data, is divided into a plurality of data blocks with massive data files and stores, make to write with reading of data and can in a plurality of data blocks, carry out simultaneously, thereby improved the access speed of data and the transmission performance of data.Simultaneously, all data that the user is needed all show after the transmission again because the present invention when the transmission data is not, but according to predetermined regular transmission data block or the partial data in the data block, reduced the time that the user waits for, improved user experience, simultaneously, because each data quantity transmitted is less, therefore can realize the strange land transmission of data.
Because when the present invention sends instruction as the user, the user instruction that sends according to client is passed to client with the result of data, rather than send the data to client and handle again by client, thereby reduced communication load, shortened user's stand-by period.
The applicant discloses a kind of embodiment that massive data files is divided into data block, each data block is compressed according to different resolution and the data of being transmitted are shown in application number is 200710017386.5 application documents, specific as follows described:
Because the data volume of massive data files is bigger, in this embodiment, during the access massive data files, massive data files is divided into a plurality of data blocks to be read and preserves, owing to divide the selection of block size to influence parallel processing capability, piecemeal is more little, divided number of a file is just many more, parallel processing capability is just strong more, data access speed is just high more, but meanwhile will increase the randomness and the piece addressing time of block access, can adopt different block sizes according to different application requirements.Concrete partition process can realize by following process: when reading massive data files, preestablish the height and the width of the data block that at every turn reads, the data block that at every turn reads is preserved.When the order read finish the data that set the deblocking width in the data line after, be stored in a memory buffer, the data that again next one set the deblocking width read up to data line and finish as being stored in another memory buffer.When the data line width can not be divided exactly by predefined deblocking width, can replenish the fill color data.The lastrow data read finishes, when the next line data are read in line feed, if the data that read in the pairing line data scope of the deblocking height that sets, then arrive the memory buffer corresponding with the lastrow data with this deposit data, otherwise are stored in the memory buffer of opening up again.
As indicated above, in order to extract the convenience of data, whole massive data files is divided into the less data block of data volume, simultaneously, for the needs of addressing, preservation comprises the locating information relevant with data block.Because each data block is occupied certain position in whole massive data files, therefore to the massive data files piecemeal time, the size and the locating information of each data block are stored, set up a data block positional structure file, spatial index file as whole massive data files tissue, this structured file can comprise resolution information, the size information of massive data files, the locating information of massive data files (start position information for example, final position information etc.), the initial locating information of each data block (for example, start position information, final position information etc.), so that when the user proposes respective request, can therefrom find out corresponding extraction data area.When massive data files is shown, locating information according to the pairing data block in viewing area, retrieve institute's data blocks stored, read in the memory buffer of client, copy to the display buffer then, data in the scope of viewing area show, and all data blocks can be spliced into a complete massive data files.
Since the size of massive data files and the size that resolution information influences data volume, and the time of the size of data volume and reading of data is closely bound up.In actual applications, the size of viewing area is fixed, various clients as a rule can be corresponding to original resolution to the query requests of mass data, but is lower than original resolution.In this case, if only there are the data of original resolution, so when handling query requests, just have at every turn all will the visit data of original resolution, the data of data volume maximum just, after taking out that part of data corresponding with the viewing area in the query requests then, by image processing techniquess such as GDI+ it is narrowed down to the size of viewing area again, such handling process obviously is a poor efficiency very.Therefore in one embodiment of the present of invention being the different client application mass data of various different resolutions all set, make client in the data of these different resolutions, to get as required.In the massive data files piecemeal, whenever the data read that sets the branch block size finish enter this memory buffer after, generate other individual-layer data of various stage resolution ratios.The hierarchy of mass data is begun by original data block, sets up the data block of a series of different resolutions, and this series data piece can reflect the detailed degree and the general picture of this data block under the various yardsticks, forms the data block of various resolution.The concrete process that data block is treated to different resolution is: can show on the display device of the client of various different resolutions in order to make massive data files, need obtain the resolution information of different clients display device in advance, after massive data files reads in memory buffer according to a minute block size, data block is carried out the preliminary treatment in early stage, comprise DC level displacement and component transformation, the result who handles is carried out wavelet transform, obtain wavelet coefficient, then wavelet coefficient is quantized, it is code block that wavelet coefficient after quantizing is divided into little data cell, each code block is carried out independently embedded encoded.With the embedded bit stream of all code blocks of obtaining,, form the data block of different quality according to rate distortion principle of optimality laminated tissue.To each layer, according to certain code stream form packing, the output compressed bit stream is preserved this code stream, discharges this memory buffer then.So repeatedly, just can each data block be compressed into the data block of different resolution size according to predefined minute block size.When the data in the client read block, according to stored parameters in required resolution dimensions information and the compressed bit stream, code stream unpacked with entropy decode, carry out inverse quantization and discrete wavelet inverse transformation then, the result of inverse transformation is carried out the synthetic mass data that obtains corresponding resolution of post-processed.
Because the resolution problem of the display device of client causes a massive data files is shown that in the visual forms of viewing area common needs roam demonstration.Location when having reduced the data demonstration by above-mentioned deblocking technology and the number of times that reads, the speed that has overcome to a certain extent in the data demonstration roam procedure is crossed slow problem.All need to read in order to solve each roam operation from hard disk, further improve the speed that data show roaming, utilize the caching technology of computer, open up a suitably memory buffer of size again at the viewing area of client display device near zone, in this buffering area, store the data that read, and then copy the appropriate section that shows in this buffer memory to viewing area.When the size of this buffering area visible forms size greater than the rolling view, then in the image roam process, system just needn't all obtain the data of display part at every turn again from the large nuber of images file, only need to calculate the position of the buffering area that should copy, and the data that try again are duplicated and got final product.The size of memory buffer can influence the efficient of data roaming, if the too little reading of data continually that then causes, too senior general is subjected to the restriction of calculator memory, and big data quantity read in the delay that also can cause speed, so memory buffer to choose more excellent mode be to select the piecemeal of close together around the viewing area.When utilizing the client video data usually, when the user pins left mouse button dragging and does not decontrol, because the scope maximum of roaming is the neighboring area of viewing area just, therefore select the nearest neighboring area in suitable branch block size and viewing area on every side as memory buffer, just the image that can realize the neighboring area, viewing area is all in the middle of memory buffer, thereby when having guaranteed roam operation, as long as hand is pinned left mouse button and do not decontroled, all roam operation do not need and read from massive data files.
At the method for above-mentioned processing mass data, a kind of system that handles mass data is disclosed in one embodiment of the present of invention, as shown in Figure 2, this system comprises data process subsystem 201, professional control subsystem 203, and client-end subsystem 202.
Data process subsystem 201 reads and preserves massive data files and locating information; The massive data files of being preserved comprises each data block of preserving by the method for processing mass data mentioned above, and the locating information of each data block correspondence, and the attribute information of preset trigger incident is preserved in first configuration file.
Professional control subsystem 203 obtains massive data files and locating information from data process subsystem 201, finish controlled function to client-end subsystem 202, when receiving user instruction, the pre-defined rule transmission data of mentioning according to the method described above, when client transmission instruction is handled the massive data files that is obtained, if described instruction is the preset trigger incident, then revises described first configuration file, and amended first configuration file is transferred to client.
Data process subsystem 201 can also become the data of different resolution with the data compression in each data block and preserves according to method mentioned above, in one embodiment of the invention, store the data block and the header file of different resolution in the massive data files of being preserved, preserve the locating information of the resolution information corresponding with the data block of described different resolution, each data block and other and massive data files and the relevant information of data block in header file, this header file is equivalent to positional structure file mentioned above.
Client-end subsystem 202, be used for the massive data files that receives is shown, can comprise communication unit, receive user instruction, usually indicate the identification information of the massive data files that current needs obtain in the user instruction, obtain massive data files to professional control subsystem 203, when receiving the data of professional control subsystem 203 transmission according to this identification information, these data are stored in memory cell, and this memory cell can be the buffer memory zone of client; After finishing receiving each data block, display unit obtains each data block according to the relevant information in the header file in the described massive data files from memory cell, and shows in corresponding position according to display packing mentioned above.
Above-mentioned data process subsystem 201 can realize by same equipment physically with client-end subsystem 202, also can realize that client-end subsystem 202 can be plurality of devices such as PDA or PC by different equipment.
Can provide video image transmission clearly by this system, realize large-scale service display function, comprise reading, historical summary playback, teaching, lecture and the report etc. of news information.
In order more clearly to understand the technical scheme in the foregoing description, below set forth the embodiment that massive data files is handled, as a kind of application example of the method and system of above-mentioned processing mass data.
The system that handles mass data among this embodiment is used to realize the making of multimedia programming, issues and perform in a radio or TV programme function.In the present embodiment, data process subsystem 201 mentioned above is finished multimedia data file is made into program file, and with the process that program file is issued, sets forth the workflow of data process subsystem 201 below in conjunction with Fig. 3:
Data process subsystem 201 reads multimedia data file according to predetermined number range piecemeal, the multi-medium data that at every turn reads is compressed preservation (step 301) successively, each data block is compressed (step 302) according to different resolution, generate positional structure file (step 303); For the ease of realizing, in the present embodiment, each data block and positional structure file are kept in the same self-defining file, the positional structure file is as the header file of this document, the resolution information of each data block, the size information of each data block (for example width of display object, height etc. have been write down in this header file, unit can be pixel), the original position of large nuber of images file and final position information (for example, origin coordinates, stop coordinate), the original position of each data block and final position information (for example, origin coordinates, stop coordinate) etc.In addition, because program file comprises a plurality of multimedia data files usually, each self-defining file that therefore will have correlation is packaged into packet, generates program file (step 304).
In addition, data process subsystem 201 also generates first configuration file, second configuration file and the 3rd configuration file (step 305).Above-mentioned configuration file can realize that for standardized needs, simultaneously because the XML resolver is more convenient, first, second, third configuration file adopts the XML script to realize in this embodiment by script file.
This first configuration file comprises and the relevant information of each program file of location (for example the identification information of each program file etc.), simultaneously, this first configuration file can also comprise the text message that program file is relevant, and the event attribute information relevant with program file.
As an embodiment, data process subsystem 201 can provide corresponding first configuration file for a plurality of professional control subsystem 203, the corresponding relation of each professional control subsystem 203 with corresponding first configuration file described, first configuration file that specified services control subsystem 203 should be loaded in second configuration file.
As mentioned above, owing to can there be a plurality of professional control subsystem 203, the identification information (for example, IP address information of professional control subsystem 203 etc.) of the professional control subsystem 203 at program file that client-end subsystem 202 need perform in a radio or TV programme and this program file place has been described in the 3rd configuration file.When strange land transmission massive data files, can be connected to the corresponding business control subsystem by the 3rd configuration file.
Data process subsystem 201 completes behind program file and the configuration file, and program file and first, second configuration file are published to professional control subsystem 203, and the 3rd configuration file is published to client-end subsystem 202 (step 306).
Professional control subsystem 203 can be positioned at server end, carries out information interaction with client-end subsystem 202, the service logic control that provides program file to perform in a radio or TV programme.Workflow below in conjunction with the professional control subsystem 203 of Fig. 4:
Professional control subsystem 203 is resolved second configuration file by the script resolver, load first configuration file and program file (step 401) according to second configuration file, when receiving the program inquiry request that client-end subsystem 202 sends, usually carry the identification information of program file and the resolution information of client-end subsystem in this program inquiry request, by resolving first configuration file, identification information according to the program file in first configuration file finds corresponding program file, transmits successively according to the identification information of each multimedia file multimedia data file with the user-defined format in the program file.In transmission during certain multimedia data file, each data block, header file and first configuration file of resolution minimum sent to client-end subsystem (step 402).
In the process that client-end subsystem 202 shows certain multimedia data file, when the user need upgrade the partial data of the multimedia data file in current shown zone, for example, certain partial data in the viewing area is amplified, dwindle or upgrade, then when professional control subsystem 203 receives the user instruction that the data to current display area change, change the positional information of data according to resolution information in the user instruction and needs in the viewing area, obtain the data block of corresponding resolution, then these data blocks are transferred to client-end subsystem 202 (step 403) according to said process.
In addition, when receiving the Event triggered information of client-end subsystem 202, whether inquiry is scheduled event according to first configuration file, if scheduled event, then the corresponding event attribute information in first configuration file is upgraded (step 404), and first configuration file after will upgrading is sent to client-end subsystem 202 (step 405).
Client-end subsystem 202 can be plurality of devices such as PDA or PC, Fig. 5 is the workflow of client-end subsystem 202: client-end subsystem 202 is resolved the 3rd configuration file by the script resolver when needed, initiate connection request (step 501) according to the 3rd configuration file to the corresponding business control subsystem, after successful connection, the information such as resolution of the program file desiring to perform in a radio or TV programme sign and client-end subsystem are sent to professional control subsystem 203 (step 502), when receiving the data of professional control subsystem 203 transmissions, the data file that receives is resolved (step 503), block copy to corresponding display region, is shown (step 504) after each data block is spliced into massive data files.
When the user need upgrade the partial data of the massive data files in current shown zone, for example, certain partial data in the viewing area is amplified, dwindled or upgrades, then send instruction, carry the resolution information that user expectation reaches in the described instruction and need the positional information of change data in the viewing area to professional control subsystem 203.
The Event triggered instruction that client-end subsystem is received the user (for example, add mark, increase background music, increase special efficacy etc.) time, send Event triggered information (step 505) to professional control subsystem 203, when receiving the control command of professional control subsystem 203, carry out relevant action (step 506) according to first configuration file that receives.
The process that client-end subsystem 202 is initiated connection request to professional control subsystem 202 in the step 501 can realize in the following manner: client-end subsystem 202 is initiated connection request by listening port to professional control subsystem 203 when starting, after professional control subsystem 203 is received this connection request, to generate random port for client-end subsystem 202, and send response to client-end subsystem 202, carry the port that generates at random in this response; After client-end subsystem 202 is received this response, send the program inquiry request and carry out follow-up information interaction by this random port.
When having a plurality of client-end subsystem, if each client-end subsystem random access is deposited the professional control subsystem of continuous multimedia messages, descend for fear of systematic function, and the problem that possibly can't realize multi-user's visit, as an embodiment, professional control subsystem sets in advance the trade off request of each client-end subsystem of scheduling mechanism, to guarantee each user's the continuity of performing in a radio or TV programme, increases the access number that can walk abreast and serve simultaneously to greatest extent.Request to each client-end subsystem, professional control subsystem need in time provide enough data, for the needs of performing in a radio or TV programme continuously, therefore, when certain client-end subsystem (has for example read enough data, read certain massive data files) after, to send message to professional control subsystem, the informing business control subsystem stops to send data, it is that this client-end subsystem is transmitted data that professional control subsystem is suspended, be other client's client-end subsystem and turn to, thereby prevent the too many system resource of data consumes of excessive buildup, even exhaust system resource.
Professional control subsystem is isolated client-end subsystem and the data process subsystem of programming in the present embodiment, is responsible for the result of forward command and incident, thus the fail safe that has improved system.In addition, when the user handles the data of current display area, can require professional control subsystem to revise corresponding event in first configuration file by send trigger event information to professional control subsystem, thereby make the user can be in real time operation such as process, handle, revise, amplify and reconfigure, thereby realize the interactivity operation the data of current viewing area.The user can pass through the program file on the real-time selection different business control subsystem of the 3rd configuration file, further improves user experience
Above-described embodiment of the present invention does not constitute the qualification to protection range of the present invention.Any modification of being done within the spirit and principles in the present invention, be equal to and replace and improvement etc., all should be included within the claim protection range of the present invention.
Claims (12)
1. a method of handling mass data is characterized in that, comprises step:
Step 1, when reading massive data files, according to predetermined number range reading of data, described number range is the scope of the data that at every turn read; The data that at every turn read are preserved, and write down the locating information of the described data that at every turn read; The attribute information of record preset trigger incident;
Step 2 when client sends instruction and obtains the massive data files of being stored, is searched needed massive data files according to the identification information in the described instruction, transmits data in the described massive data files according to pre-defined rule; When described client transmission instruction is handled the massive data files that is obtained, if described instruction is the preset trigger incident, then revise the attribute information of relative trigger incident, and amended attribute information is transferred to described client.
2. method according to claim 1 is characterized in that, also comprises step: each data compression of reading is become the data of different resolution, the data of described different resolution are preserved, and write down corresponding resolution information;
Described pre-defined rule comprises: according to the resolution information in the described instruction data of corresponding resolution are transmitted, and the scope of the data of each transmission is consistent with the described number range of reading of data.
3. method according to claim 2, it is characterized in that, when the instruction of described client is the preset trigger incident, described pre-defined rule also comprises: when the instruction of described trigger event for the data of current display area are handled, transmit the data of corresponding resolution according to the positional information of resolution information in the described instruction and viewing area.
4. method according to claim 2 is characterized in that described locating information, resolution information are stored in the positional structure file.
5. method according to claim 1 is characterized in that, described step 1 also comprises: the massive data files that is read is packaged into packet, and writes down the identification information of described packet.
6. method according to claim 5 is characterized in that, the attribute information of described preset trigger incident and the identification information storage of described packet are in first configuration file, and described first configuration file is a script file.
7. a system that handles mass data is characterized in that, comprising: data process subsystem and professional control subsystem;
Described data process subsystem is used to read and preserves massive data files and locating information; And the attribute information of preset trigger incident preserved in first configuration file;
The massive data files of being preserved comprises at least one data block, and each data block comprises the data that read according to predetermined number range respectively, and described number range is the scope of the data that at every turn read; Described locating information is the locating information of described data block correspondence;
Described professional control subsystem, be used to obtain described massive data files, locating information and described first configuration file, when the massive data files of being stored is obtained in client transmission instruction, search needed massive data files according to the identification information in the described instruction, transmit data in the described massive data files according to pre-defined rule; When described client transmission instruction is handled the massive data files that is obtained, if described instruction is the preset trigger incident, then revise described first configuration file, and amended first configuration file is transferred to described client.
8. system according to claim 7 is characterized in that, described data process subsystem also is used for data compression with described data block and becomes the data of different resolution and preserve,
Described massive data files also comprises the data of different resolution and the resolution information corresponding with the data of described different resolution;
Described pre-defined rule is: according to the resolution information in the described instruction data of corresponding resolution are transmitted.
9. system according to claim 8 is characterized in that, described data process subsystem also is used to generate second configuration file, and described second configuration file is published to described professional control subsystem;
Described second configuration file is used to preserve the correspondence relationship information of the described professional control subsystem and first configuration file;
Described professional control subsystem is obtained corresponding first configuration file according to described second configuration file.
10. system according to claim 9, it is characterized in that this system also comprises client-end subsystem, be used to receive user instruction, obtain massive data files according to identification information in the described user instruction, show according to the locating information in the described massive data files.
11. system according to claim 10 is characterized in that, described data process subsystem also is used to generate the 3rd configuration file, and described the 3rd configuration file is published to described client-end subsystem;
Described the 3rd configuration file is used to preserve the IP address information of described professional control subsystem and the identification information of described packet,
Described client-end subsystem is initiated connection request according to described IP address information to described professional control subsystem, and carries out information interaction according to the identification information and the described professional control subsystem of described packet, obtains data.
12. system according to claim 11 is characterized in that, described first configuration file, second configuration file, the 3rd configuration file are script file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2007100302529A CN101119278A (en) | 2007-09-14 | 2007-09-14 | Method and system for processing mass data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2007100302529A CN101119278A (en) | 2007-09-14 | 2007-09-14 | Method and system for processing mass data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101119278A true CN101119278A (en) | 2008-02-06 |
Family
ID=39055223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2007100302529A Pending CN101119278A (en) | 2007-09-14 | 2007-09-14 | Method and system for processing mass data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101119278A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143154A (en) * | 2010-12-28 | 2011-08-03 | 华为技术有限公司 | Method for preventing attack on media server and media server |
| CN102521383A (en) * | 2011-12-22 | 2012-06-27 | 南京烽火星空通信发展有限公司 | Method for storing and accessing mass files in distributed system |
| CN101350837B (en) * | 2008-08-28 | 2012-07-04 | 深圳市迅雷网络技术有限公司 | Method and system for identifying file package bundling |
| CN104050200A (en) * | 2013-03-15 | 2014-09-17 | 伊姆西公司 | Method and device for copying data |
| CN105354326A (en) * | 2015-11-26 | 2016-02-24 | 航天恒星科技有限公司 | Spatial position indexing method and system |
| CN105843819A (en) * | 2015-01-15 | 2016-08-10 | 中国移动通信集团河南有限公司 | Method and device for exporting data |
| CN107870928A (en) * | 2016-09-26 | 2018-04-03 | 上海泓智信息科技有限公司 | File reading and device |
| CN108170455A (en) * | 2018-03-12 | 2018-06-15 | 晶晨半导体(上海)股份有限公司 | The packaging method and upgrade method of upgrade package |
| CN104199816B (en) * | 2008-05-14 | 2019-04-09 | 起元技术有限责任公司 | The management storage of independent accessible data unit |
| CN109714650A (en) * | 2018-12-18 | 2019-05-03 | 联想(北京)有限公司 | A kind of data processing method and device |
| CN109814926A (en) * | 2018-12-28 | 2019-05-28 | 东软集团股份有限公司 | A kind of method and apparatus for extracting data |
| CN110019501A (en) * | 2017-08-24 | 2019-07-16 | 深圳市金证科技股份有限公司 | A kind of collecting method, device and terminal device |
| CN112153089A (en) * | 2019-06-27 | 2020-12-29 | 上海梅山钢铁股份有限公司 | Method for reading and writing OPC data block |
-
2007
- 2007-09-14 CN CNA2007100302529A patent/CN101119278A/en active Pending
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104199816B (en) * | 2008-05-14 | 2019-04-09 | 起元技术有限责任公司 | The management storage of independent accessible data unit |
| CN101350837B (en) * | 2008-08-28 | 2012-07-04 | 深圳市迅雷网络技术有限公司 | Method and system for identifying file package bundling |
| CN102143154B (en) * | 2010-12-28 | 2015-12-09 | 华为技术有限公司 | For method and the media server of preventing attack on media server |
| CN102143154A (en) * | 2010-12-28 | 2011-08-03 | 华为技术有限公司 | Method for preventing attack on media server and media server |
| CN102521383A (en) * | 2011-12-22 | 2012-06-27 | 南京烽火星空通信发展有限公司 | Method for storing and accessing mass files in distributed system |
| CN104050200A (en) * | 2013-03-15 | 2014-09-17 | 伊姆西公司 | Method and device for copying data |
| CN104050200B (en) * | 2013-03-15 | 2017-12-08 | 伊姆西公司 | Method and apparatus for data copy |
| CN105843819A (en) * | 2015-01-15 | 2016-08-10 | 中国移动通信集团河南有限公司 | Method and device for exporting data |
| CN105354326A (en) * | 2015-11-26 | 2016-02-24 | 航天恒星科技有限公司 | Spatial position indexing method and system |
| CN105354326B (en) * | 2015-11-26 | 2020-09-04 | 航天恒星科技有限公司 | Spatial position indexing method and system |
| CN107870928A (en) * | 2016-09-26 | 2018-04-03 | 上海泓智信息科技有限公司 | File reading and device |
| CN110019501A (en) * | 2017-08-24 | 2019-07-16 | 深圳市金证科技股份有限公司 | A kind of collecting method, device and terminal device |
| CN108170455A (en) * | 2018-03-12 | 2018-06-15 | 晶晨半导体(上海)股份有限公司 | The packaging method and upgrade method of upgrade package |
| CN109714650A (en) * | 2018-12-18 | 2019-05-03 | 联想(北京)有限公司 | A kind of data processing method and device |
| CN109714650B (en) * | 2018-12-18 | 2021-05-18 | 联想(北京)有限公司 | Data processing method and device |
| CN109814926A (en) * | 2018-12-28 | 2019-05-28 | 东软集团股份有限公司 | A kind of method and apparatus for extracting data |
| CN109814926B (en) * | 2018-12-28 | 2021-07-09 | 东软集团股份有限公司 | Method and device for extracting data |
| CN112153089A (en) * | 2019-06-27 | 2020-12-29 | 上海梅山钢铁股份有限公司 | Method for reading and writing OPC data block |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101119278A (en) | Method and system for processing mass data | |
| CN100505630C (en) | A method and system for broadcasting buffered advertisement in polling mode under the network living broadcast environment | |
| CN105144722B (en) | Network coded storage with multi-resolution codes | |
| CN103828339B (en) | Splicing for digital image search | |
| CN101594518B (en) | Distributed on-demand media transcoding system and method | |
| CN101127578A (en) | A method and system for processing a magnitude of data | |
| CN103944864A (en) | Information transmission method, information sending device, receiving device and system | |
| CN103609117B (en) | Code and decode the method and device of image | |
| CN1941899A (en) | System and method for downloading contents | |
| CN102404567A (en) | System and method for transcoding network monitoring video stream in real time as required | |
| CN101577627A (en) | System and method for downloading and playing multimedia file | |
| CN110096245A (en) | Distributed Synergy user interface and application projection | |
| CN102497353A (en) | Processing method, server and system for multi-server distributed data | |
| CN105430534A (en) | Intelligent device reporting data method and system | |
| CN101039372B (en) | Apparatus and system for managing television album | |
| KR102686703B1 (en) | System and method for cross-media reporting through high-speed merging of data sources | |
| CN109862381A (en) | Promotion method, system, device and storage medium based on dynamic video poster | |
| CN101312522A (en) | Video play-on-demand system | |
| CN104639985A (en) | Multimedia playing control method and system | |
| CN101656947B (en) | Establishing method, device and system of cross-heterogeneous-network service sharing | |
| CN108563704A (en) | A kind of method, apparatus and equipment of data write-in | |
| US20120134599A1 (en) | Method and system for providing tile map service using solid compression | |
| CN109040640A (en) | A kind of distributed video storage-playback | |
| US9253281B2 (en) | Cells and/or vantage points in streaming media | |
| CN105791964B (en) | cross-platform media file playing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20080206 |