CN113626277A - Collector communication system, method and device in supercomputing environment - Google Patents
Collector communication system, method and device in supercomputing environment Download PDFInfo
- Publication number
- CN113626277A CN113626277A CN202110740442.XA CN202110740442A CN113626277A CN 113626277 A CN113626277 A CN 113626277A CN 202110740442 A CN202110740442 A CN 202110740442A CN 113626277 A CN113626277 A CN 113626277A
- Authority
- CN
- China
- Prior art keywords
- data
- analyzer
- filtered
- preset
- collector communication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004891 communication Methods 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000001914 filtration Methods 0.000 claims abstract description 39
- 230000002688 persistence Effects 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 13
- 230000008569 process Effects 0.000 claims abstract description 10
- 238000004458 analytical method Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 5
- 230000003993 interaction Effects 0.000 claims description 3
- 238000011084 recovery Methods 0.000 description 6
- 238000007405 data analysis Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 238000012216 screening Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
- G06F11/3093—Configuration details thereof, e.g. installation, enabling, spatial arrangement of the probes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a collector communication system, a collector communication method and a collector communication device under an ultra-computing environment, wherein the method comprises the following steps: receiving data through a network card and directly sending the data to a data filter; the data filter reads a preset filtering rule once every preset interval time, starts a filtering process according to the preset filtering rule to filter data, and sends the filtered data to the data acquisition unit; judging whether the filtered data is sensitive data or not through a data acquisition unit, and if so, sending the filtered data to an authentication module; if the data is non-sensitive data, sending the filtered data to a data analyzer; the authentication module encrypts the filtered data by adopting a preset encryption mode, generates encrypted data after encryption is completed and sends the encrypted data to the data analyzer; the data analyzer performs data persistence on the received data. The invention greatly enhances the data processing capacity and the safety of the infrastructure management platform in the cluster of the super-computation scene.
Description
Technical Field
The invention relates to the technical field of communication, in particular to a collector communication system, a collector communication method and a collector communication device in a super computing environment.
Background
In the supercomputing environment, through mutual communication among the collector clusters of the infrastructure management platform, other network packets in supercomputing are prevented from influencing collection, and the performance of the clusters is analyzed. In order to ensure the best performance among the nodes, the nodes are configured with a secret-free login and a firewall is closed, so that the management nodes such as the Slurm and the like can freely execute commands among the nodes to carry out scheduling. And the nodes are free from being secret, so that the mutual communication among the collector clusters can be greatly influenced.
In the traditional field, each node is limited by a firewall, so that the number of network packets which need to be processed really is not large, but in the super computing field, because massive data are generated after running between the nodes, if the collector and the infrastructure management platform are communicated and data transmission is carried out in the traditional interface mode, the interface can possibly receive other data between clusters, JAVA continuously creates objects, the recovery frequency of a GC is improved, and abnormal service is caused.
Disclosure of Invention
In view of the above problems, an object of the present invention is to provide a collector communication system, method and apparatus in a supercomputing environment.
In order to achieve the purpose, the invention is realized by the following technical scheme: a collector communication system in a supercomputing environment, comprising: the authentication system comprises a network card, a data filter, a data acquisition unit, an authentication module and a data analyzer, wherein the data acquisition unit is respectively connected with the data filter, the authentication module and the data analyzer; the network card is used for receiving data and directly sending the data to the data filter; the data filter is used for reading a preset filtering rule, filtering data by using the filtering rule and sending the filtered data to the data acquisition unit; the data acquisition unit judges whether the received data is sensitive data, if the received data is the sensitive data, the received data is sent to the authentication module for authentication and then sent to the data analyzer, and if the received data is non-sensitive data, the received data is directly sent to the data analyzer; the data acquisition unit performs data interaction with the data analyzer in a UDP mode, and performs data processing and data transmission in a byte mode; the data analyzer is used for decrypting the received data and performing data persistence.
Correspondingly, the invention also discloses a collector communication method under the supercomputing environment, which comprises the following steps:
s1: receiving data through a network card and directly sending the data to a data filter;
s2: the data filter reads a preset filtering rule once every preset interval time, refreshes a buffer area storing the preset filtering rule, starts a filtering flow according to the preset filtering rule to filter data, and sends the filtered data to the data acquisition unit;
s3: judging whether the filtered data is sensitive data or not through a data acquisition unit, and if so, sending the filtered data to an authentication module; if the data is non-sensitive data, sending the filtered data to a data analyzer;
s4: the authentication module encrypts the filtered data by adopting a preset encryption mode, generates encrypted data after encryption is completed and sends the encrypted data to the data analyzer;
s5: the data analyzer performs data persistence on the received data.
Further, the filtering process in step S2 includes:
searching whether a first separator exists in the data received by the data filter through a first instruction;
if not, directly discarding the received data;
if the character string with the first separator exists, the character string with the first separator in the data is divided, the character string before the first separator is stored in the variable A, and the character string after the first separator is stored in the variable B;
and matching the character string in the variable A with a preset rule base, and if the character string can be matched with the preset rule base, taking the corresponding data as filtered data and sending the filtered data to a data acquisition unit.
Further, the step S3 includes:
in the data acquisition unit, dividing the character string in the variable B by using a second separator through a second instruction, and storing the character string before the second separator in the variable C;
reading the value of the variable C, if the variable C is 00, the filtered data is non-sensitive data, and directly sending the filtered data to a data analyzer; and if the variable C is 01, the filtered data is sensitive data, and the filtered data is sent to the authentication module.
Further, the preset encryption mode supports configuration file configuration, an MD5 encryption algorithm and an AES encryption algorithm, and the MD5 encryption algorithm is used for encryption 2048 times by default; if the encryption mode is changed, the authentication module sends a broadcast to the data analyzer, and the data analyzer updates the decryption mode after receiving the notification.
Further, the step S5 includes:
after receiving the encrypted data, the data analyzer decrypts the encrypted data according to the current decryption mode, and stores the data into a database according to code logic for persistence after taking out the data;
and after receiving the non-sensitive data, the data analyzer directly stores the non-sensitive data into a preset database according to the code logic for persistence.
Further, the preset database adopts an Influxdb database or a Mysql database.
Further, the preset rule base stores Java character strings represented by 16-system ASCII codes.
Further, the preset rule base stores:
ASCII code 16 424 d 43 to represent BMC;
16-ary ASCII code 54455945 for representing a feature analysis system;
16 ASCII code 495350494 d to represent an infrastructure management platform.
Correspondingly, the invention discloses a collector communication device under an ultra-computation environment, which comprises:
the memory is used for storing the collector communication program in the super computing environment;
and the processor is used for realizing the steps of the collector communication method in the supercomputing environment when executing the collector communication program in the supercomputing environment.
Correspondingly, the invention discloses a readable storage medium, wherein the readable storage medium is stored with a collector communication program in a super-computing environment, and the collector communication program in the super-computing environment is executed by a processor to realize the steps of the collector communication method in the super-computing environment.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention designs a reasonable data filter and an authentication module, so that the infrastructure management platform clusters can normally interact under the condition of large-scale network throughput in a supercomputing scene, and the subsequent function expansion can be facilitated according to a custom filtering rule base.
2. According to the invention, data are filtered through an extensible filtering rule, effective data screening is completed very quickly, the screened data can be processed by the authentication module, and quick data analysis can be performed according to classification.
3. The invention greatly enhances the data processing capacity and the security of the infrastructure management platform in the cluster of the super-computation scene (non-security), and the whole flow of data is transmitted in a byte mode, thereby effectively reducing JAVA continuously created objects and reducing the recovery frequency of GC.
Therefore, compared with the prior art, the invention has prominent substantive features and remarkable progress, and the beneficial effects of the implementation are also obvious.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a system block diagram of the present invention;
FIG. 2 is a flow chart of the method of the present invention.
In the figure, 1 is a network card; 2 is a data filter; 3 is a data acquisition unit; 4 is an authentication module; and 5, a data analyzer.
Detailed Description
The core of the invention is to provide a collector communication method under an ultra-computation environment, in the prior art, in the ultra-computation field, due to the massive data after running operation between nodes, if the collector and the infrastructure management platform are communicated and also adopt the traditional interface mode to carry out data transmission, the interface can possibly receive other data between clusters, JAVA continuously creates objects, the recovery frequency of GC is improved, and abnormal service is caused.
According to the collector communication method under the super-computing environment, firstly, data are transmitted into the network card and then are filtered according to the filtering rule. And the data enters a data acquisition unit after being filtered, and the data acquisition unit determines whether the data is sensitive data according to analysis. If the sensitive data are sensitive data, entering a data analyzer after being authenticated by an authentication module; and if the data is non-sensitive data, directly entering a data analyzer. And finally, performing data persistence through a data analyzer. In addition, the data full flow is transmitted in a byte mode, so that the whole process reduces JAVA continuous creation objects and reduces the recovery frequency of GC. Therefore, the data processing capacity and the safety of the infrastructure management platform in the cluster of the super-computation scene (non-safety) can be effectively improved.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
as shown in fig. 1, the present embodiment provides a collector communication system in a supercomputing environment, including: the system comprises a network card 1, a data filter 2, a data collector 3, an authentication module 4 and a data analyzer 5. The data acquisition device 3 is respectively connected with the data filter 2, the authentication module 4 and the data analyzer 5, the network card 1 is connected with the data filter 2, and the authentication module 4 is connected with the data analyzer 5.
The network card 1 is used for receiving data and directly sending the data to the data filter 2; the data filter 3 is used for reading a preset filtering rule, filtering data by using the filtering rule, and sending the filtered data to the data acquisition unit 3; the data acquisition unit 3 judges whether the received data is sensitive data, if the received data is sensitive data, the received data is sent to the authentication module 4 for authentication and then sent to the data analyzer 5, and if the received data is non-sensitive data, the received data is directly sent to the data analyzer 5; the data acquisition unit 3 performs data interaction with the data analyzer 5 in a UDP mode, and performs data processing and data transmission in a byte mode; the data analyzer 5 is used for decrypting and data persisting the received data.
The embodiment provides a collector communication system in a supercomputing environment, which designs a reasonable data filter and an authentication module, so that under the condition of large-scale network throughput in a supercomputing scene, an infrastructure management platform cluster can be normally interacted, and the subsequent function expansion is facilitated according to a custom filtering rule base.
Example two:
based on the first embodiment, as shown in fig. 2, the invention also discloses a collector communication method in a supercomputing environment, which comprises the following steps:
s1: and receiving data through the network card and directly sending the data to the data filter.
S2: the data filter reads the preset filtering rules once every preset interval time, refreshes the buffer area storing the preset filtering rules, starts the filtering process according to the preset filtering rules to filter the data, and sends the filtered data to the data acquisition unit.
Wherein the preset interval time is five minutes. The filtering process comprises the following steps:
firstly, searching whether a first separator exists in data received by a data filter through a first instruction; if not, directly discarding the received data; if the character string with the first separator exists, the character string in the data is divided, the character string before the first separator is stored in the variable A, and the character string after the first separator is stored in the variable B. And then, matching the character string in the variable A with a preset rule base, and if the character string can be matched with the preset rule base, taking the corresponding data as filtered data and sending the filtered data to a data acquisition unit.
The preset rule base stores Java character strings represented by 16-system ASCII codes, such as the 16-system ASCII code 424 d 43, which is used for representing the BMC; 16-ary ASCII code 54455945 for representing a feature analysis system; 16 ASCII code 495350494 d to represent an infrastructure management platform.
S3: judging whether the filtered data is sensitive data or not through a data acquisition unit, and if so, sending the filtered data to an authentication module; and if the data is non-sensitive data, sending the filtered data to a data analyzer.
The method specifically comprises the following steps: in the data collector, the character string in the variable B is divided by the second delimiter through the second instruction, and the character string before the second delimiter is stored in the variable C. Reading the value of the variable C, if the variable C is 00, the filtered data is non-sensitive data, and directly sending the filtered data to a data analyzer; and if the variable C is 01, the filtered data is sensitive data, and the filtered data is sent to the authentication module.
S4: the authentication module encrypts the filtered data by adopting a preset encryption mode, and generates encrypted data after encryption and sends the encrypted data to the data analyzer.
The preset encryption mode supports configuration file configuration, an MD5 encryption algorithm and an AES encryption algorithm, and the MD5 encryption algorithm is used for encryption 2048 times by default; if the encryption mode is changed, the authentication module sends a broadcast to the data analyzer, and the data analyzer updates the decryption mode after receiving the notification.
S5: the data analyzer performs data persistence on the received data.
The method comprises the following specific steps: after receiving the encrypted data, the data analyzer decrypts the encrypted data according to the current decryption mode, and stores the data into a database according to code logic for persistence after taking out the data; and after receiving the non-sensitive data, the data analyzer directly stores the non-sensitive data into a preset database according to the code logic for persistence.
The preset database adopts an Influxdb database or a Mysql database.
The embodiment provides a collector communication method in a supercomputing environment, data is filtered through an extensible filtering rule, effective data screening is completed very quickly, screened data can be processed by an authentication module to obtain sensitive data, and quick data analysis can be performed according to classification. The method greatly enhances the data processing capacity and the safety of the infrastructure management platform in the cluster of the supercomputing scene (non-safety), and the whole flow of data is transmitted in a byte mode, so that JAVA continuously created objects can be effectively reduced, and the recovery frequency of GC is reduced.
Example three:
based on the above embodiment, this embodiment also discloses a collector communication method in a supercomputing environment, including:
1. the network card directly enters a data filter after receiving mass data;
2. the data filter reads the filtering rule once every 5 minutes and refreshes the cache region of the rule; the filtration process is as follows:
A. the first lookup looks for the separator "-" by instruction 2d (-) and if not, drops directly.
B. If the separator exists, dividing the character string with the separator, storing the character string before the separator in the array [0], and storing the character string after the separator in the array [1 ]; and matching the array [0] with a rule base, and entering matched data into a data acquisition unit.
The current rule base:
424 d 43(BMC) data sent to the ISPIM infrastructure management platform on behalf of the BMC;
54455945 (TEYE) represents the data sent by the T-eye application profiling system to ISPIM.
3. Data entering the data collector is divided into array [1] using string 495350494 d (representing ISPIM), and the string preceding "ISPIM" in the string of array [1] is stored in array [0 ].
Reading the value of the array [0 ]; if the value is 00, the representative does not need to be encrypted and directly enters a data analyzer; if the number is 01, the user needs to encrypt and enters the authentication module.
4. The authentication module is responsible for encrypting the data to be encrypted in the data acquisition unit and then transmitting the encrypted data to the data analyzer; the encryption mode supports profile configuration, support (MD5, AES), and encryption 2048 times using MD5 by default. Once the encryption mode is changed, the authentication module sends a broadcast to all analyzers, and the analyzers update the decryption mode after receiving the notification.
5. And after receiving the data, the data analyzer decrypts the data according to the current decryption mode, takes out the data and stores the data into Influxdb or Mysql according to code logic for persistence.
The embodiment provides a collector communication method in a supercomputing environment, data is filtered through an extensible filtering rule, effective data screening is completed very quickly, screened data can be processed by an authentication module to obtain sensitive data, and quick data analysis can be performed according to classification. The method greatly enhances the data processing capacity and the safety of the infrastructure management platform in the cluster of the supercomputing scene (non-safety), and the whole flow of data is transmitted in a byte mode, so that JAVA continuously created objects can be effectively reduced, and the recovery frequency of GC is reduced.
Example four:
the embodiment discloses a collector communication device in a supercomputing environment, which comprises a processor and a memory; wherein, the processor implements the following steps when executing the collector communication program in the supercomputing environment stored in the memory:
1. and after the data are transmitted into the network card, the data are filtered according to the filtering rule.
2. The data enters a data acquisition unit after being filtered, and the data acquisition unit determines to enter step 3 or step 4 according to whether the analysis is sensitive data;
3. the sensitive data enters a data analyzer after being authenticated by the authentication module;
4. the data goes directly to the data analyzer.
5. The data analyzer persists.
6. The data is transmitted in a byte mode in the whole flow.
Further, the collector communication device in the supercomputing environment in this embodiment may further include:
and the input interface is used for acquiring the collector communication program under the super-computing environment imported from the outside, storing the acquired collector communication program under the super-computing environment into the memory, and also used for acquiring various instructions and parameters transmitted by external terminal equipment and transmitting the instructions and parameters to the processor, so that the processor performs corresponding processing by using the instructions and the parameters. In this embodiment, the input interface may specifically include, but is not limited to, a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, and the like.
And the output interface is used for outputting various data generated by the processor to the terminal equipment connected with the output interface, so that other terminal equipment connected with the output interface can acquire various data generated by the processor. In this embodiment, the output interface may specifically include, but is not limited to, a USB interface, a serial interface, and the like.
And the communication unit is used for establishing remote communication connection between the collector communication device and an external server in the supercomputing environment, so that the collector communication device in the supercomputing environment can mount the mirror image file into the external server. In this embodiment, the communication unit may specifically include, but is not limited to, a remote communication unit based on a wireless communication technology or a wired communication technology.
And the keyboard is used for acquiring various parameter data or instructions input by a user through real-time key cap knocking.
And the display is used for displaying relevant information in the short circuit positioning process of the power supply line of the running server in real time.
The mouse can be used for assisting a user in inputting data and simplifying the operation of the user.
Example five:
the present embodiments also disclose a readable storage medium, where the readable storage medium includes Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable hard disk, a CD-ROM, or any other form of storage medium known in the art. The readable storage medium stores a collector communication program in a super computing environment, and when the collector communication program in the super computing environment is executed by a processor, the collector communication program in the super computing environment realizes the following steps:
1. and after the data are transmitted into the network card, the data are filtered according to the filtering rule.
2. The data enters a data acquisition unit after being filtered, and the data acquisition unit determines to enter step 3 or step 4 according to whether the analysis is sensitive data;
3. the sensitive data enters a data analyzer after being authenticated by the authentication module;
4. the data goes directly to the data analyzer.
5. The data analyzer persists.
6. The data is transmitted in a byte mode in the whole flow.
In conclusion, the invention designs the reasonable data filter and the authentication module, so that the infrastructure management platform cluster can normally interact under the condition of large-scale network throughput in the supercomputing scene, and the subsequent function expansion can be facilitated according to the custom filtering rule base. According to the invention, data are filtered through an extensible filtering rule, effective data screening is completed very quickly, the screened data can be processed by the authentication module, and quick data analysis can be performed according to classification.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The method disclosed by the embodiment corresponds to the system disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the description of the method part.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided by the present invention, it should be understood that the disclosed system, system and method can be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, systems or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit.
Similarly, each processing unit in the embodiments of the present invention may be integrated into one functional module, or each processing unit may exist physically, or two or more processing units are integrated into one functional module.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The collector communication system, method, device and readable storage medium under the supercomputing environment provided by the invention are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. A collector communication system in a supercomputing environment, comprising: a network card, a data filter, a data collector, an authentication module and a data analyzer,
the data acquisition unit is respectively connected with the data filter, the authentication module and the data analyzer, the network card is connected with the data filter, and the authentication module is connected with the data analyzer;
the network card is used for receiving data and directly sending the data to the data filter; the data filter is used for reading a preset filtering rule, filtering data by using the filtering rule and sending the filtered data to the data acquisition unit; the data acquisition unit judges whether the received data is sensitive data, if the received data is the sensitive data, the received data is sent to the authentication module for authentication and then sent to the data analyzer, and if the received data is non-sensitive data, the received data is directly sent to the data analyzer; the data acquisition unit performs data interaction with the data analyzer in a UDP mode, and performs data processing and data transmission in a byte mode; the data analyzer is used for decrypting the received data and performing data persistence.
2. A collector communication method under an ultra-computing environment is characterized by comprising the following steps:
s1: receiving data through a network card and directly sending the data to a data filter;
s2: the data filter reads a preset filtering rule once every preset interval time, refreshes a buffer area storing the preset filtering rule, starts a filtering flow according to the preset filtering rule to filter data, and sends the filtered data to the data acquisition unit;
s3: judging whether the filtered data is sensitive data or not through a data acquisition unit, and if so, sending the filtered data to an authentication module; if the data is non-sensitive data, sending the filtered data to a data analyzer;
s4: the authentication module encrypts the filtered data by adopting a preset encryption mode, generates encrypted data after encryption is completed and sends the encrypted data to the data analyzer;
s5: the data analyzer performs data persistence on the received data.
3. The collector communication method in the supercomputing environment according to claim 2, wherein the filtering process in the step S2 includes:
searching whether a first separator exists in the data received by the data filter through a first instruction;
if not, directly discarding the received data;
if the character string with the first separator exists, the character string with the first separator in the data is divided, the character string before the first separator is stored in the variable A, and the character string after the first separator is stored in the variable B;
and matching the character string in the variable A with a preset rule base, and if the character string can be matched with the preset rule base, taking the corresponding data as filtered data and sending the filtered data to a data acquisition unit.
4. The collector communication method in the supercomputing environment according to claim 3, wherein the step S3 includes:
in the data acquisition unit, dividing the character string in the variable B by using a second separator through a second instruction, and storing the character string before the second separator in the variable C;
reading the value of the variable C, if the variable C is 00, the filtered data is non-sensitive data, and directly sending the filtered data to a data analyzer; and if the variable C is 01, the filtered data is sensitive data, and the filtered data is sent to the authentication module.
5. The collector communication method in the supercomputing environment according to claim 4, wherein the preset encryption mode supports configuration file configuration, supports an MD5 encryption algorithm and an AES encryption algorithm, and encrypts 2048 times by default using an MD5 encryption algorithm; if the encryption mode is changed, the authentication module sends a broadcast to the data analyzer, and the data analyzer updates the decryption mode after receiving the notification.
6. The collector communication method in the supercomputing environment according to claim 4, wherein the step S5 includes:
after receiving the encrypted data, the data analyzer decrypts the encrypted data according to the current decryption mode, and stores the data into a database according to code logic for persistence after taking out the data;
and after receiving the non-sensitive data, the data analyzer directly stores the non-sensitive data into a preset database according to the code logic for persistence.
7. The collector communication method in the supercomputing environment according to claim 6, wherein the preset database is an infiluxdb database or a Mysql database.
8. The collector communication method in the supercomputing environment according to claim 3, wherein a Java string represented by 16 ASCII code is stored in the preset rule base.
9. The collector communication method in the supercomputing environment according to claim 8, wherein the preset rule base stores:
ASCII code 16 424 d 43 to represent BMC;
16-ary ASCII code 54455945 for representing a feature analysis system;
16 ASCII code 495350494 d to represent an infrastructure management platform.
10. A collector communication device in a supercomputing environment, comprising:
the memory is used for storing the collector communication program in the super computing environment;
a processor, configured to implement the steps of the collector communication method in the supercomputing environment according to any one of claims 2 to 9 when executing the collector communication program in the supercomputing environment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110740442.XA CN113626277A (en) | 2021-06-30 | 2021-06-30 | Collector communication system, method and device in supercomputing environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110740442.XA CN113626277A (en) | 2021-06-30 | 2021-06-30 | Collector communication system, method and device in supercomputing environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113626277A true CN113626277A (en) | 2021-11-09 |
Family
ID=78378735
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110740442.XA Pending CN113626277A (en) | 2021-06-30 | 2021-06-30 | Collector communication system, method and device in supercomputing environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113626277A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080127125A1 (en) * | 2006-10-27 | 2008-05-29 | Microsoft Corporation | Virtualization For Diversified Tamper Resistance |
| US20130111220A1 (en) * | 2011-10-31 | 2013-05-02 | International Business Machines Corporation | Protecting sensitive data in a transmission |
| CN112325933A (en) * | 2020-10-29 | 2021-02-05 | 贵州电网有限责任公司 | Multi-energy system acquisition and preprocessing method and terminal based on edge calculation |
| CN112487444A (en) * | 2020-11-25 | 2021-03-12 | 远光软件股份有限公司 | Database-based data encryption method and device, storage medium and electronic equipment |
-
2021
- 2021-06-30 CN CN202110740442.XA patent/CN113626277A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080127125A1 (en) * | 2006-10-27 | 2008-05-29 | Microsoft Corporation | Virtualization For Diversified Tamper Resistance |
| US20130111220A1 (en) * | 2011-10-31 | 2013-05-02 | International Business Machines Corporation | Protecting sensitive data in a transmission |
| CN112325933A (en) * | 2020-10-29 | 2021-02-05 | 贵州电网有限责任公司 | Multi-energy system acquisition and preprocessing method and terminal based on edge calculation |
| CN112487444A (en) * | 2020-11-25 | 2021-03-12 | 远光软件股份有限公司 | Database-based data encryption method and device, storage medium and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| SENSITIVE DATA FILTERING.ET AL: "Big data query optimization by using Locality Sensitive Bloom Filter", IEEE, 4 May 2015 (2015-05-04) * |
| 李自清;: "基于网络的数据库敏感数据加密模型研究", 计算机测量与控制, no. 05, 25 May 2017 (2017-05-25) * |
| 白昱阳;黄彦浩;陈思远;张俊;李柏青;王飞跃;: "云边智能:电力系统运行控制的边缘计算方法及其应用现状与展望", 自动化学报, no. 03, 15 March 2020 (2020-03-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113542253B (en) | Network flow detection method, device, equipment and medium | |
| US9912689B2 (en) | Anonymized network data collection and network threat assessment and monitoring systems and methods | |
| CN113591119B (en) | Cross-domain identification analysis node data privacy protection and safety sharing method and system | |
| CN110855676B (en) | Network attack processing method and device and storage medium | |
| Mughaid et al. | Improved dropping attacks detecting system in 5g networks using machine learning and deep learning approaches | |
| CN112511512A (en) | Vulnerability scanning engine and risk management system of threat detection engine | |
| CN113923057B (en) | Data processing method and device for satellite measurement, operation and control platform, electronic equipment and medium | |
| CN106340104B (en) | A kind of coded lock, coded lock control system and method | |
| US20210329479A1 (en) | Network Analytics | |
| CN112468520A (en) | Data detection method, device and equipment and readable storage medium | |
| CN111191247A (en) | Database security audit system | |
| CN110554877A (en) | JSON data analysis method, device, equipment and storage medium | |
| CN113518042B (en) | Data processing method, device, equipment and storage medium | |
| CN114338171A (en) | Black product attack detection method and device | |
| CN105207829B (en) | Intrusion detection data processing method, device and system | |
| CN112445861A (en) | Information processing method, device, system and storage medium | |
| CN113626277A (en) | Collector communication system, method and device in supercomputing environment | |
| CN111737091B (en) | Log processing method and device and readable medium | |
| Liu et al. | Smart hardware hybrid secure searchable encryption in cloud with IoT privacy management for smart home system | |
| CN107548058B (en) | Equipment access method and intelligent terminal | |
| WO2017036042A1 (en) | Information collection method and apparatus | |
| CN109660541B (en) | Method and device for acquiring decentralized big data | |
| KR102682233B1 (en) | How to configure electronic test instruments and optional features | |
| CN115484326A (en) | Method, system and storage medium for processing data | |
| CN117201021B (en) | Key dynamic switching method, system, device and medium for main service system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |