CN115664931A - Alarm data association method, device, storage medium and equipment - Google Patents
Alarm data association method, device, storage medium and equipment Download PDFInfo
- Publication number
- CN115664931A CN115664931A CN202211272977.XA CN202211272977A CN115664931A CN 115664931 A CN115664931 A CN 115664931A CN 202211272977 A CN202211272977 A CN 202211272977A CN 115664931 A CN115664931 A CN 115664931A
- Authority
- CN
- China
- Prior art keywords
- alarm data
- alarm
- risk
- target
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Alarm Systems (AREA)
Abstract
The application discloses a method, a device, a storage medium and equipment for associating alarm data, wherein for each alarm data, the alarm data is preprocessed to obtain target alarm data; extracting the characteristics of the target alarm data to obtain target alarm characteristics; inputting the target alarm characteristics into a noise identification model to obtain an identification result output by the noise identification model; when the score of the target alarm characteristic is not lower than the preset score, identifying the target alarm data as high-risk alarm data; according to risk characteristics contained in the high-risk alarm data, grouping the high-risk alarm data layer by layer from a root node to a leaf node in a preset decision tree to obtain grouped high-risk alarm data; according to the preset fields to be merged, the grouped high-risk alarm data are compressed to obtain a high-risk alarm set.
Description
Technical Field
The present application relates to the field of alarm data, and in particular, to a method, an apparatus, a storage medium, and a device for associating alarm data.
Background
With the development of emerging technologies such as cloud computing and 5G, the attack surface of a network space is greatly expanded. Meanwhile, the attack skills of attackers are rapidly evolving. In order to effectively capture threat behaviors, various levels of network space agents (such as supervision departments) are provided with various types of network security detection and defense equipment in network spaces, and can generate alarms aiming at different types of data and risks so as to analyze alarm data subsequently.
At present, a Chinese-style data lake Management mode is generally adopted, alarm data is analyzed through a Security Information and Event Management (SIEM) platform, the SIEM platform needs to perform steps of de-duplication, disambiguation, classification, suppression and the like on the alarm data in a data lake, alarm suppression and association aggregation are realized, and due to the adoption of excessive single alarm characteristic dimensions (such as based on a source IP and a threshold), large-scale alarm suppression combination and association aggregation are performed, deep level characteristic dimensions in a dynamic alarm data stream are difficult to effectively extract, and therefore the alarm data association is poor.
Therefore, how to improve the relevance before the alarm data becomes an urgent problem to be solved in the field.
Disclosure of Invention
The application provides a method, a device, a storage medium and equipment for associating alarm data, and aims to improve the association of the alarm data.
In order to achieve the above object, the present application provides the following technical solutions:
a method for associating alarm data comprises the following steps:
for each alarm data, preprocessing the alarm data to obtain target alarm data; the alarm data is obtained from a database in advance;
extracting the characteristics of the target alarm data to obtain target alarm characteristics; the target alarm data is non-low-quality alarm data;
inputting the target alarm characteristics into a noise recognition model to obtain a recognition result output by the noise recognition model; the identification result at least comprises the score of the target alarm characteristic;
when the score of the target alarm characteristic is not lower than a preset score, identifying the target alarm data as high-risk alarm data;
according to the risk characteristics contained in the high-risk alarm data, grouping the high-risk alarm data layer by layer from a root node to a leaf node in a preset decision tree to obtain grouped high-risk alarm data;
and compressing the grouped high-risk alarm data according to a preset field to be merged to obtain a high-risk alarm set.
Optionally, the training process of the noise identification model includes:
acquiring alarm data in a preset historical time period from the database, and identifying the alarm data as historical alarm data;
sequencing the historical alarm data according to the time sequence from morning to evening to obtain a historical alarm sequence; the historical alarm sequence comprises each historical alarm data;
mapping the historical alarm data by using a preset mapping rule for each piece of historical alarm data to obtain first alarm data;
filtering the first alarm data to obtain second alarm data;
extracting the characteristics of the second alarm data to obtain a characteristic training sequence; the feature training sequence comprises each feature sample;
inputting each feature sample into an initial model, and coding each feature sample through a coder of the initial model to obtain each feature vector output by the initial model;
and taking each feature vector as the input of a task loss function of the initial model, calculating to obtain the score of the historical alarm data, continuously adjusting each parameter of the task loss function until the loss function of the noise recognition model is converged, and confirming that the noise recognition model is successfully trained.
Optionally, the method further includes:
sequencing the compressed high-risk alarm data according to the sequence of the high risk degree from high to low to obtain a high-risk alarm sequence; the high-risk alarm sequence at least comprises the sequence bits of each compressed high-risk alarm data;
for each compressed high-risk alarm data, sequentially acquiring link information of the compressed high-risk alarm data according to the sequence from small to large of sequence order, and sending the link information of the compressed high-risk alarm data to a user so as to enable the user to analyze;
when processing information sent by the user is received, acquiring a response processing strategy corresponding to the high-risk alarm set from a response knowledge base; the processing information indicates information for processing the high-risk alarm set;
and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the high-risk alarm set.
Optionally, the preprocessing the alarm data to obtain target alarm data includes:
and for each alarm data, mapping the alarm data by using a preset mapping rule to obtain target alarm data.
Optionally, the method further includes:
when the score of the target alarm characteristic is lower than the preset score, identifying the target alarm data as noise alarm data;
compressing the noise alarm data and the low-quality alarm data by using a preset merging rule to obtain compressed noise alarm data and low-quality alarm data;
and merging the compressed noise alarm data and the low-quality alarm data by utilizing statistical aggregation to obtain a low-quality noise alarm set.
Optionally, the method further includes:
acquiring a response processing strategy corresponding to the low-quality noise alarm set from a response knowledge base;
and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the low-quality noise alarm set.
Optionally, the method further includes:
and acquiring source information of each alarm data in the low-quality noise alarm set, and performing feature extraction on the source information according to a preset proportion to obtain each piece of relevant information.
An apparatus for associating alarm data, comprising:
the processing unit is used for preprocessing the alarm data to obtain target alarm data; the alarm data is obtained from a database in advance;
the extraction unit is used for extracting the characteristics of the target alarm data to obtain target alarm characteristics; the target alarm data is non-low-quality alarm data;
the input unit is used for inputting the target alarm characteristics into a noise identification model to obtain an identification result output by the noise identification model; the identification result at least comprises the score of the target alarm characteristic;
the identification unit is used for identifying the target alarm data as high-risk alarm data when the score of the target alarm characteristic is not lower than a preset score;
the grouping unit is used for grouping the high-risk alarm data layer by layer from a root node to a leaf node in a preset decision tree according to the risk characteristics contained in the high-risk alarm data to obtain grouped high-risk alarm data;
and the compression unit is used for compressing the grouped high-risk alarm data according to the preset field to be merged to obtain a high-risk alarm set.
A computer-readable storage medium comprising a stored program, wherein the program, when executed by a processor, performs the method of associating alarm data.
An apparatus for associating alarm data, comprising: a processor, memory, and a bus; the processor and the memory are connected through the bus;
the memory is used for storing a program, and the processor is used for executing the program, wherein the program is executed by the processor by the association method of the alarm data.
According to the technical scheme, for each alarm data, the alarm data are preprocessed to obtain target alarm data; extracting the characteristics of the target alarm data to obtain target alarm characteristics; inputting the target alarm characteristics into a noise identification model to obtain an identification result output by the noise identification model; when the score of the target alarm characteristic is not lower than a preset score, identifying the target alarm data as high-risk alarm data; according to risk characteristics contained in the high-risk alarm data, grouping the high-risk alarm data layer by layer from a root node to a leaf node in a preset decision tree to obtain grouped high-risk alarm data; according to the preset fields to be merged, the grouped high-risk alarm data are compressed to obtain a high-risk alarm set.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1a is a flowchart of an association method of alarm data according to an embodiment of the present application;
fig. 1b is a flowchart of an association method of alarm data according to an embodiment of the present application;
fig. 2 is a flowchart of a training method of a noise identification model according to an embodiment of the present disclosure;
fig. 3 is a flowchart of another alarm data association method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an apparatus for associating alarm data according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As shown in fig. 1a and fig. 1b, a flowchart of an association method of alarm data provided in an embodiment of the present application includes:
s101: and acquiring each alarm data from the database.
The types of the alarm data include, but are not limited to: multi-source alarm logs, multi-type alarm logs.
S102: and mapping the alarm data by using a preset mapping rule for each alarm data to obtain target alarm data.
The specific implementation manner of mapping the alarm data by using the preset mapping rule is common knowledge of those skilled in the art, and is not described herein again.
Optionally, the preset mapping rule may be selected according to an actual situation, and is not limited specifically here.
S103: and judging whether the target alarm data is non-low-quality alarm data or not.
If the target alarm data is non-low quality alarm data, executing S104, otherwise executing S108.
Wherein, the low-quality alarm data is: the alarm caused by the lack of the alarm characteristic field can not be judged and correlated.
It should be noted that, the specific implementation process of determining whether the target alarm data is non-low-quality alarm data is as follows: identifying the target alarm data by using a preset identification method; the identification method includes but is not limited to missing value identification, error field type identification, and error field range identification.
S104: and extracting the characteristics of the target alarm data to obtain the target alarm characteristics.
The specific implementation manner of performing feature extraction on the target alarm data is common knowledge of people in the art and is not described herein again.
S105: and inputting the target alarm characteristics into the noise identification model to obtain an identification result output by the noise identification model.
Wherein the identification result at least comprises the score of the target alarm characteristic.
It should be noted that the noise identification model is trained in advance based on the historical alarm data, and in particular, the training process of the noise identification model may refer to the steps shown in fig. 2 and the explanation of the steps.
Optionally, the noise identification model includes, but is not limited to, a self-encoder model.
S106: and judging whether the score of the target alarm characteristic is lower than a preset score or not.
If the score of the target alarm characteristic is lower than the preset score, executing S107, otherwise executing S110.
Specifically, assuming that the preset score is 80 and the score of the target alarm feature is 70, it is determined whether the score of the target alarm feature is lower than the preset score, and obviously, the score of the target alarm feature is lower than the preset score, and therefore, S107 is continuously performed.
Specifically, assuming that the preset score is 80 and the score of the target alarm feature is 90, it is determined whether the score of the target alarm feature is lower than the preset score, and obviously, the score of the target alarm feature is not lower than the preset score, and therefore, S110 is continuously performed.
S107: the target alert data is identified as noise alert data.
S108: and compressing the noise alarm data and the low-quality alarm data by using a preset merging rule to obtain the compressed noise alarm data and the compressed low-quality alarm data.
The preset merge rule includes, but is not limited to: a higher compression ratio squash merge rule.
It should be noted that, the specific implementation manner of compressing the noise alarm data and the low-quality alarm data by using the preset merging rule is common knowledge of those skilled in the art, and is not described herein again.
It is emphasized that, since the quantity of the low-quality alarm data and the noise alarm data is large in scale but the threat information is not large, in order to reduce the quantity of the alarm data, the noise alarm data and the low-quality alarm data are compressed by using the preset merging rule.
S109: and merging the compressed noise alarm data and the low-quality alarm data by utilizing statistical aggregation to obtain a low-quality noise alarm set.
After execution of S109, execution of S117 is continued.
Optionally, the content included in the low quality noise alarm set may be: "low quality: missing key fields "," low quality: error field type identification "," noise: webbql injection attempt "," noise: mySQL login attempt ".
S110: and identifying the target alarm data as high-risk alarm data.
S111: and according to the risk characteristics contained in the high-risk alarm data, grouping the high-risk alarm data layer by layer from a root node to a leaf node in a preset decision tree to obtain grouped high-risk alarm data.
It should be noted that, for the leaf nodes on the same layer of the same parent node, sorting is performed according to different features of the current layer, and the sorting order may be determined according to configuration (for example, "high risk attack type 1" has a higher priority than "high risk attack type 2", and "success" has a higher priority than "failure").
S112: and compressing the grouped high-risk alarm data according to the preset to-be-merged field to obtain a high-risk alarm set.
The high-risk alarm set comprises compressed high-risk alarm data, and the types of the preset fields to be merged include but are not limited to: basic object attribute information such as an IP address, a port number, an alarm type and a domain name, threat description information such as a threat level, an attack stage and an attack success mark, and asset vulnerability information corresponding to the alarm.
Optionally, the grouped high-risk alarm data may be compressed according to a preset field to be merged, so as to obtain a network security event.
Specifically, assuming that a preset field to be merged is a source IP, and a target IP, according to the preset field to be merged, compressing the grouped high-risk alarm data, and obtaining a high-risk alarm set as follows: type (2): successful discrimination of high risk attack type 1and attack: successful and alert noise score: low and associated vulnerability level: high and warning frequency: above 10 times, attack source IP list: [ { ip1:10 times }, { ip2:6 } and destination Domain name List: { domain:1 time }, { domain2:20 },. Wherein "and" is used only to denote a cascade.
S113: and sequencing the compressed high-risk alarm data according to the sequence of the high-risk degree to the low-risk degree to obtain a high-risk alarm sequence.
And the high-risk alarm sequence at least comprises the sequence bits of each compressed high-risk alarm data.
S114: and for each compressed high-risk alarm data, sequentially acquiring the link information of the compressed high-risk alarm data according to the sequence from small to large, and sending the link information of the compressed high-risk alarm data to a user so as to analyze the high-risk alarm data by the user.
S115: and when receiving the processing information sent by the user, acquiring a response processing strategy corresponding to the high-risk alarm set from the response knowledge base.
Wherein, the processing information indicates to process the information of the high-risk alarm set, and the response processing strategy includes but is not limited to: the method comprises the steps of blocking related source addresses, sending short message mail notification, and generating a vulnerability correction work order and a security event work order.
S116: and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the high-risk alarm set.
S117: and acquiring a response processing strategy corresponding to the low-quality noise alarm set from a response knowledge base.
When S117 is executed, S119 may be executed concurrently.
S118: and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the low-quality noise alarm set.
S119: and acquiring source information of each alarm data in the low-quality noise alarm set, and performing feature extraction on the source information according to a preset proportion to obtain each piece of relevant information.
The source information includes, but is not limited to, generating device, generating API.
In conclusion, the compressed noise alarm data and the low-quality alarm data are merged by utilizing statistical aggregation to obtain a low-quality noise alarm set; compressing the grouped high-risk alarm data according to a preset field to be merged to obtain a high-risk alarm set; compared with the prior art, the alarm data of different types are subjected to suppression combination and aggregation association according to different dimensions, and the association between the alarm data is improved.
As shown in fig. 2, a flowchart of a training method for a noise identification model provided in the embodiment of the present application includes:
s201: and acquiring alarm data in a preset historical time period from the database, and identifying the alarm data as historical alarm data.
S202: and sequencing the historical alarm data according to the time sequence from morning to night to obtain a historical alarm sequence.
Wherein the historical alarm sequence comprises various historical alarm data.
It should be noted that, the historical alarm data is sorted, so as to analyze the attack path, attack surface, and influence range of the alarm data.
S203: and mapping the historical alarm data by using a preset mapping rule for each piece of historical alarm data to obtain first alarm data.
The specific implementation manner of mapping the historical data by using the preset mapping rule is common knowledge of those skilled in the art, and is not described herein again.
It should be noted that, the historical alarm data is mapped by using a preset mapping rule, so as to align the fields of different types of alarms uniformly, thereby forming alarm data with uniform fields.
S204: and filtering the first alarm data to obtain second alarm data.
The specific implementation process of filtering the first alarm data is as follows: filtering the alarm data which do not accord with the preset conditions in the first alarm data; the preset conditions are as follows: the field is missing, the alarm data which can not support the study and judgment, and the alarm data caused by the known attack test and the safe operation and maintenance operation are not needed.
S205: and extracting the characteristics of the second alarm data to obtain a characteristic training sequence.
The feature training sequence includes each feature sample, and the type of each feature sample includes but is not limited to: basic attribute information such as an IP address, a port number and an alarm type, threat description information such as a threat level, an attack stage and an attack success mark, and asset vulnerability information corresponding to the alarm.
S206: and inputting each characteristic sample into the initial model, and coding each characteristic sample through a coder of the initial model to obtain each characteristic vector output by the initial model.
Wherein the initial model includes an encoder and a task loss function.
It should be noted that the type of the initial model used in the present embodiment may be an auto-encoder model.
S207: and taking each feature vector as the input of the task loss function of the initial model, calculating to obtain the score of the historical alarm data, continuously adjusting each parameter of the task loss function until the loss function of the noise identification model is converged, and confirming that the noise identification model is successfully trained.
In summary, by using the scheme shown in this embodiment, the noise identification model can be obtained through effective training.
As shown in fig. 3, a flowchart of another method for associating alarm data provided in the embodiment of the present application includes:
s301: and for each alarm data, preprocessing the alarm data to obtain target alarm data.
Wherein, the alarm data is obtained from the database in advance.
S302: and extracting the characteristics of the target alarm data to obtain the target alarm characteristics.
Wherein the target alarm data is non-low quality alarm data.
S303: and inputting the target alarm characteristics into the noise identification model to obtain an identification result output by the noise identification model.
Wherein the identification result at least comprises the score of the target alarm characteristic.
S304: and when the score of the target alarm characteristic is not lower than the preset score, identifying the target alarm data as high-risk alarm data.
S305: and according to the risk characteristics contained in the high-risk alarm data, grouping the high-risk alarm data layer by layer from a root node to a leaf node in a preset decision tree to obtain grouped high-risk alarm data.
S306: and compressing the grouped high-risk alarm data according to the preset to-be-merged field to obtain a high-risk alarm set.
In conclusion, the compressed noise alarm data and the low-quality alarm data are merged by utilizing statistical aggregation to obtain a low-quality noise alarm set; compressing the grouped high-risk alarm data according to a preset field to be merged to obtain a high-risk alarm set; compared with the prior art, the method and the device have the advantages that different types of alarm data are subjected to pressing combination and aggregation association according to different dimensions, and the association between the alarm data is improved.
As shown in fig. 4, an architecture diagram of an apparatus for associating alarm data provided in the embodiment of the present application includes:
the processing unit 100 is configured to perform preprocessing on the alarm data to obtain target alarm data; the alarm data is obtained from the database in advance.
The processing unit 100 is specifically configured to: and mapping the alarm data by using a preset mapping rule for each alarm data to obtain target alarm data.
The extraction unit 200 is configured to perform feature extraction on the target alarm data to obtain target alarm features; the target alarm data is non-low quality alarm data.
The input unit 300 is configured to input the target alarm characteristic into the noise identification model to obtain an identification result output by the noise identification model; the recognition result at least comprises the score of the target alarm characteristic.
The input unit 300 is specifically configured to: acquiring alarm data in a preset historical time period from a database, and identifying the alarm data as historical alarm data; sequencing the historical alarm data according to the time sequence from morning to evening to obtain a historical alarm sequence; the historical alarm sequence comprises various historical alarm data; mapping the historical alarm data by using a preset mapping rule for each piece of historical alarm data to obtain first alarm data; filtering the first alarm data to obtain second alarm data; extracting the characteristics of the second alarm data to obtain a characteristic training sequence; the characteristic training sequence comprises each characteristic sample; inputting each characteristic sample into an initial model, and coding each characteristic sample through a coder of the initial model to obtain each characteristic vector output by the initial model; and taking each feature vector as the input of the task loss function of the initial model, calculating to obtain the score of the historical alarm data, continuously adjusting each parameter of the task loss function until the loss function of the noise identification model is converged, and confirming that the noise identification model is successfully trained.
And the identification unit 400 is used for identifying the target alarm data as high-risk alarm data when the score of the target alarm characteristic is not lower than the preset score.
The identification unit 400 is further configured to identify the target alarm data as noise alarm data when the score of the target alarm characteristic is lower than a preset score; compressing the noise alarm data and the low-quality alarm data by using a preset merging rule to obtain compressed noise alarm data and low-quality alarm data; and merging the compressed noise alarm data and the low-quality alarm data by utilizing statistical aggregation to obtain a low-quality noise alarm set.
The identifying unit 400 is further configured to obtain a response processing policy corresponding to the low-quality noise alarm set from a response knowledge base; and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the low-quality noise alarm set.
The identifying unit 400 is further configured to obtain source information of each alarm data in the low-quality noise alarm set, and perform feature extraction on the source information according to a preset proportion to obtain each piece of relevant information.
And the grouping unit 500 is configured to group the high-risk alarm data layer by layer from a root node to a leaf node in the preset decision tree according to the risk characteristics included in the high-risk alarm data, so as to obtain the grouped high-risk alarm data.
And a compressing unit 600, configured to compress the grouped high-risk alarm data according to the preset field to be merged, to obtain a high-risk alarm set.
The compression unit 600 is further configured to sequence the compressed high-risk alarm data according to a sequence from high to low in risk degree, so as to obtain a high-risk alarm sequence; the high-risk alarm sequence at least comprises the sequence bits of each high-risk alarm data after compression; for each compressed high-risk alarm data, sequentially acquiring link information of the compressed high-risk alarm data according to the sequence from small to large, and sending the link information of the compressed high-risk alarm data to a user so as to analyze the high-risk alarm data by the user; when processing information sent by a user is received, acquiring a response processing strategy corresponding to the high-risk alarm set from a response knowledge base; the processing information indicates to process the information of the high-risk alarm set; and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the high-risk alarm set.
In conclusion, the compressed noise alarm data and the low-quality alarm data are merged by utilizing statistical aggregation to obtain a low-quality noise alarm set; compressing the grouped high-risk alarm data according to a preset field to be merged to obtain a high-risk alarm set; compared with the prior art, the method and the device have the advantages that different types of alarm data are subjected to pressing combination and aggregation association according to different dimensions, and the association between the alarm data is improved.
The application also provides a computer readable storage medium, which comprises a stored program, wherein the program executes the association method of the alarm data provided by the application.
The present application further provides an associated device of alarm data, including: a processor, a memory, and a bus. The processor is connected with the memory through a bus, the memory is used for storing programs, and the processor is used for running the programs, wherein when the programs are run, the association method of the alarm data provided by the application is executed, and the association method comprises the following steps:
for each alarm data, preprocessing the alarm data to obtain target alarm data; the alarm data is obtained from a database in advance;
extracting the characteristics of the target alarm data to obtain target alarm characteristics; the target alarm data is non-low-quality alarm data;
inputting the target alarm characteristics into a noise identification model to obtain an identification result output by the noise identification model; the identification result at least comprises the score of the target alarm characteristic;
when the score of the target alarm characteristic is not lower than a preset score, identifying the target alarm data as high-risk alarm data;
according to risk characteristics contained in the high-risk alarm data, grouping the high-risk alarm data layer by layer from a root node to a leaf node in a preset decision tree to obtain grouped high-risk alarm data;
and compressing the grouped high-risk alarm data according to a preset field to be merged to obtain a high-risk alarm set.
Optionally, the training process of the noise identification model includes:
acquiring alarm data in a preset historical time period from the database, and identifying the alarm data as historical alarm data;
sequencing the historical alarm data according to the time sequence from morning to evening to obtain a historical alarm sequence; the historical alarm sequence comprises each historical alarm data;
mapping the historical alarm data by using a preset mapping rule for each piece of historical alarm data to obtain first alarm data;
filtering the first alarm data to obtain second alarm data;
extracting the characteristics of the second alarm data to obtain a characteristic training sequence; the feature training sequence comprises each feature sample;
inputting each feature sample into an initial model, and coding each feature sample through a coder of the initial model to obtain each feature vector output by the initial model;
and taking each feature vector as the input of a task loss function of the initial model, calculating to obtain the score of the historical alarm data, continuously adjusting each parameter of the task loss function until the loss function of the noise recognition model is converged, and confirming that the noise recognition model is successfully trained.
Optionally, the method further includes:
sequencing the compressed high-risk alarm data according to the sequence from high risk degree to low risk degree to obtain a high-risk alarm sequence; the high-risk alarm sequence at least comprises the sequence bits of each compressed high-risk alarm data;
for each compressed high-risk alarm data, sequentially acquiring link information of the compressed high-risk alarm data according to the sequence from small to large of sequence order, and sending the link information of the compressed high-risk alarm data to a user so as to enable the user to analyze;
when processing information sent by the user is received, acquiring a response processing strategy corresponding to the high-risk alarm set from a response knowledge base; the processing information indicates information for processing the high-risk alarm set;
and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the high-risk alarm set.
Optionally, the preprocessing the alarm data to obtain target alarm data includes:
and for each alarm data, mapping the alarm data by using a preset mapping rule to obtain target alarm data.
Optionally, the method further includes:
when the score of the target alarm characteristic is lower than the preset score, identifying the target alarm data as noise alarm data;
compressing the noise alarm data and the low-quality alarm data by using a preset merging rule to obtain compressed noise alarm data and low-quality alarm data;
and merging the compressed noise alarm data and the low-quality alarm data by utilizing statistical aggregation to obtain a low-quality noise alarm set.
Optionally, the method further includes:
acquiring a response processing strategy corresponding to the low-quality noise alarm set from a response knowledge base;
and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the low-quality noise alarm set.
Optionally, the method further includes:
and acquiring source information of each alarm data in the low-quality noise alarm set, and performing feature extraction on the source information according to a preset proportion to obtain each piece of relevant information.
The functions described in the method of the embodiment of the present application, if implemented in the form of software functional units and sold or used as independent products, may be stored in a storage medium readable by a computing device. Based on such understanding, part of the technical solutions or portions of the embodiments contributing to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computing device (which may be a personal computer, a server, a mobile computing device, a network device, or the like) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: u disk, removable hard disk, read only memory, random access memory, magnetic or optical disk, etc. for storing program codes.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for associating alarm data is characterized by comprising the following steps:
for each alarm data, preprocessing the alarm data to obtain target alarm data; the alarm data is obtained from a database in advance;
extracting the characteristics of the target alarm data to obtain target alarm characteristics; the target alarm data is non-low-quality alarm data;
inputting the target alarm characteristics into a noise identification model to obtain an identification result output by the noise identification model; the identification result at least comprises the score of the target alarm characteristic;
when the score of the target alarm characteristic is not lower than a preset score, identifying the target alarm data as high-risk alarm data;
according to the risk characteristics contained in the high-risk alarm data, grouping the high-risk alarm data layer by layer from a root node to a leaf node in a preset decision tree to obtain grouped high-risk alarm data;
and compressing the grouped high-risk alarm data according to a preset field to be merged to obtain a high-risk alarm set.
2. The method of claim 1, wherein the training process of the noise recognition model comprises:
acquiring alarm data in a preset historical time period from the database, and identifying the alarm data as historical alarm data;
sequencing the historical alarm data according to the time sequence from morning to evening to obtain a historical alarm sequence; the historical alarm sequence comprises each historical alarm data;
mapping the historical alarm data by using a preset mapping rule for each piece of historical alarm data to obtain first alarm data;
filtering the first alarm data to obtain second alarm data;
extracting the characteristics of the second alarm data to obtain a characteristic training sequence; the feature training sequence comprises each feature sample;
inputting each feature sample into an initial model, and coding each feature sample through a coder of the initial model to obtain each feature vector output by the initial model;
and taking each feature vector as the input of a task loss function of the initial model, calculating to obtain the score of the historical alarm data, continuously adjusting each parameter of the task loss function until the loss function of the noise recognition model is converged, and confirming that the noise recognition model is successfully trained.
3. The method of claim 1, further comprising:
sequencing the compressed high-risk alarm data according to the sequence of the high risk degree from high to low to obtain a high-risk alarm sequence; the high-risk alarm sequence at least comprises the sequence bits of each compressed high-risk alarm data;
for each compressed high-risk alarm data, sequentially acquiring link information of the compressed high-risk alarm data according to the sequence from small to large of sequence order, and sending the link information of the compressed high-risk alarm data to a user so as to enable the user to analyze;
when processing information sent by the user is received, acquiring a response processing strategy corresponding to the high-risk alarm set from a response knowledge base; the processing information indicates information for processing the high-risk alarm set;
and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the high-risk alarm set.
4. The method of claim 1, wherein the preprocessing the alarm data for each alarm data to obtain target alarm data comprises:
and for each alarm data, mapping the alarm data by using a preset mapping rule to obtain target alarm data.
5. The method of claim 1, further comprising:
when the score of the target alarm characteristic is lower than the preset score, identifying the target alarm data as noise alarm data;
compressing the noise alarm data and the low-quality alarm data by using a preset merging rule to obtain compressed noise alarm data and low-quality alarm data;
and merging the compressed noise alarm data and the low-quality alarm data by utilizing statistical aggregation to obtain a low-quality noise alarm set.
6. The method of claim 5, further comprising:
acquiring a response processing strategy corresponding to the low-quality noise alarm set from a response knowledge base;
and calling an interface corresponding to the response handling strategy to respond according to the response handling strategy corresponding to the low-quality noise alarm set.
7. The method of claim 5, further comprising:
and acquiring source information of each alarm data in the low-quality noise alarm set, and performing feature extraction on the source information according to a preset proportion to obtain each piece of relevant information.
8. An apparatus for associating alarm data, comprising:
the processing unit is used for preprocessing the alarm data to obtain target alarm data; the alarm data is obtained from a database in advance;
the extraction unit is used for extracting the characteristics of the target alarm data to obtain target alarm characteristics; the target alarm data is non-low quality alarm data;
the input unit is used for inputting the target alarm characteristics into a noise identification model to obtain an identification result output by the noise identification model; the identification result at least comprises the score of the target alarm characteristic;
the identification unit is used for identifying the target alarm data as high-risk alarm data when the score of the target alarm characteristic is not lower than a preset score;
the grouping unit is used for grouping the high-risk alarm data layer by layer from a root node to a leaf node in a preset decision tree according to the risk characteristics contained in the high-risk alarm data to obtain grouped high-risk alarm data;
and the compression unit is used for compressing the grouped high-risk alarm data according to the preset field to be merged to obtain a high-risk alarm set.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program, when executed by a processor, performs the method of associating alarm data according to any one of claims 1-7.
10. An apparatus for associating alarm data, comprising: a processor, a memory, and a bus; the processor and the memory are connected through the bus;
the memory is adapted to store a program and the processor is adapted to execute the program, wherein the program when executed by the processor performs the method of associating alarm data according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211272977.XA CN115664931A (en) | 2022-10-18 | 2022-10-18 | Alarm data association method, device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211272977.XA CN115664931A (en) | 2022-10-18 | 2022-10-18 | Alarm data association method, device, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115664931A true CN115664931A (en) | 2023-01-31 |
Family
ID=84990031
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211272977.XA Pending CN115664931A (en) | 2022-10-18 | 2022-10-18 | Alarm data association method, device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115664931A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118555186A (en) * | 2024-07-26 | 2024-08-27 | 中国铁塔股份有限公司 | Method, device, equipment and storage medium for compressing alarm data |
-
2022
- 2022-10-18 CN CN202211272977.XA patent/CN115664931A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118555186A (en) * | 2024-07-26 | 2024-08-27 | 中国铁塔股份有限公司 | Method, device, equipment and storage medium for compressing alarm data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108881263B (en) | Network attack result detection method and system | |
| CN108471429B (en) | Network attack warning method and system | |
| CN108683687B (en) | Network attack identification method and system | |
| Ektefa et al. | Intrusion detection using data mining techniques | |
| CN112819336B (en) | Quantification method and system based on network threat of power monitoring system | |
| CN111355697B (en) | Detection method, device, equipment and storage medium for botnet domain name family | |
| CN113676464A (en) | Network security log alarm processing method based on big data analysis technology | |
| CN108833185B (en) | Network attack route restoration method and system | |
| CN112153062B (en) | Multi-dimension-based suspicious terminal equipment detection method and system | |
| CN109218321A (en) | A kind of network inbreak detection method and system | |
| CN112839014A (en) | Method, system, device and medium for establishing model for identifying abnormal visitor | |
| CN111723371A (en) | Method for constructing detection model of malicious file and method for detecting malicious file | |
| CN113704328B (en) | User behavior big data mining method and system based on artificial intelligence | |
| CN110365636B (en) | Method and device for judging attack data source of industrial control honeypot | |
| CN115242441A (en) | Network intrusion detection method based on feature selection and deep neural network | |
| CN115664931A (en) | Alarm data association method, device, storage medium and equipment | |
| CN111885011B (en) | Method and system for analyzing and mining safety of service data network | |
| CN118041587A (en) | Network security test evaluation system and method | |
| CN116668054A (en) | Security event collaborative monitoring and early warning method, system, equipment and medium | |
| CN115828245A (en) | Malicious file identification method based on deep learning | |
| CN114205146A (en) | Processing method and device for multi-source heterogeneous security log | |
| CN113055396B (en) | Cross-terminal traceability analysis method, device, system and storage medium | |
| CN112751863A (en) | Attack behavior analysis method and device | |
| CN112597498A (en) | Webshell detection method, system and device and readable storage medium | |
| CN118487872B (en) | Nuclear power industry-oriented network abnormal behavior detection and analysis method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |