KR20230166933A - System for filtering false positive data based on log-based user's abnormal behavior and feature attribution - Google Patents

Abstract

The present invention relates to a log-based user's abnormal behavior detection and a filtering system for false positive data based on feature contribution, which performs the function of determining whether test data is positive or negative by a pre-learned anomaly detection model. detection unit; and a filtering unit that classifies test data determined as positive by the abnormality detection unit as false positive or true positive by a pre-learned filtering model and removes false positive test data. Detection of abnormal behavior by a user based on logs. Provides a filtering system for false positive data based on feature contribution.

Description

Log-based user abnormal behavior detection and filtering system for false positive data based on feature contribution {SYSTEM FOR FILTERING FALSE POSITIVE DATA BASED ON LOG-BASED USER'S ABNORMAL BEHAVIOR AND FEATURE ATTRIBUTION}

The present invention relates to a system for detecting abnormal user behavior based on logs and filtering false positive data based on feature contributions. More specifically, the present invention relates to a system for detecting abnormal user behavior in log data through machine learning or deep learning and based on feature contributions. This relates to a filtering system for false positive data based on log-based user abnormal behavior detection and feature contribution that can efficiently and accurately remove false positive data.

Anomaly detection is a technique to identify rare items, events, or observations that significantly deviate from most typical data. The most commonly used methods include unsupervised learning-based methods in which no labels exist in the training data.

Unsupervised learning-based anomaly detection is a method that assumes that the majority of data in an unlabeled data set is normal and detects data points that are different from normal data. However, because abnormal data points are very rare and heterogeneous, it is difficult to find all abnormal phenomena, so there is a problem of high false positive (FP) rate, which judges negative (normal) data as positive (abnormal) data in actual data. .

In addition, for this problem, if it is rule-based anomaly detection, it is possible to understand the results to some extent based on the rules, but if a complex model is used to improve the performance of the anomaly detection model, it is difficult to understand the model's judgment results and why the model is used. There is also a problem that it is difficult to understand whether negative (normal) data was judged as positive (abnormality).

The present invention is intended to solve the problems described above, and provides a log-based user system that detects abnormal user behavior based on log data and efficiently and accurately removes false positive data from the detection results based on feature contribution. The purpose is to provide a system for detecting abnormal behavior and filtering false positive data based on feature contribution.

In order to solve the problems described above, the present invention is a log-based user's abnormal behavior detection and filtering system for false positive data based on feature contribution, and determines whether test data is positive or negative by a pre-learned anomaly detection model. An abnormality detection unit that performs a judgment function; and a filtering unit that classifies test data determined as positive by the abnormality detection unit as false positive or true positive by a pre-learned filtering model and removes false positive test data. Detection of abnormal behavior by a user based on logs. Provides a filtering system for false positive data based on feature contribution.

Here, the anomaly detection unit includes a preprocessing unit that performs preprocessing on input data; And it may include an anomaly detection model that performs learning for anomaly detection based on learning data among the input data and outputs a result of whether it is positive or negative for the inspection data among the input data.

Additionally, the input data may include inspection data, which is data that is the target of anomaly detection, training data used to train an anomaly detection model, and evaluation data used to train a filtering model.

In addition, the data for which preprocessing has been completed by the preprocessor can be classified into normal data and abnormal data, the normal data can be divided and included in the learning data and evaluation data, and the abnormal data can be included only in the evaluation data.

In addition, the filtering unit inputs the evaluation data into the anomaly detection unit, labels the evaluation data determined as positive by the anomaly detection unit as true positive and false positive, and transfers the labeled evaluation data to the feature contribution generation unit. a labeling generation unit that transmits; a feature contribution generator that generates contribution information for predicting each feature as positive for input data, and replaces each feature with the generated contribution information to generate secondary features; and is learned by secondary features generated by the feature contribution generation unit for evaluation data, and after learning is completed, the secondary features generated by the feature contribution generation unit are used for inspection data determined to be positive by the anomaly detection unit. It may include a filtering model that receives secondary features, classifies false positives or true positives, and removes false positive test data.

In addition, the input data input to the feature contribution generation unit may be evaluation data labeled by the labeling generation unit when learning a filtering model, and may be test data judged positive by the anomaly detection unit when detecting an anomaly. .

In addition, when performing filtering model learning, the feature contribution generation unit generates information on the contribution of each feature of the evaluation data determined to be positive by the anomaly detection unit labeled by the labeling generation unit to predicting it as positive, , Generate secondary features by replacing each feature with the generated contribution information, and when detecting an anomaly, generate contribution information for predicting that each feature of the test data determined to be positive by the anomaly detection unit is positive. , secondary features can be created by replacing each feature with the generated contribution information.

In addition, the feature contribution information generation unit calculates the difference between model prediction values when each feature is included and when each feature is not included for each feature used while the input data passes through the anomaly detection model into a subset of all possible features. By calculating the contribution information for each feature, secondary features can be created by replacing existing features with the generated contribution information.

According to the present invention, log-based user abnormal behavior detection and feature contribution-based detection of user abnormal behavior can efficiently and accurately remove false positive data from detection results based on feature contribution while detecting user abnormal behavior based on log data. A filtering system for false positive data can be provided.

Figure 1 is a diagram showing the overall configuration of a system 100 for detecting abnormal behavior of users based on logs and filtering false positive data based on feature contributions according to the present invention.
Figure 2 is a diagram showing the configuration of the abnormality detection unit 10.
Figure 3 is a diagram showing the configuration of the filtering unit 20.
Figure 4 is a diagram showing an example of secondary features generated by contribution information.
Figure 5 shows measurement results evaluating the performance of the system 100 according to the present invention.
Figure 6 shows a comparison of the ROC-Curve before and after applying the filtering unit 20.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

Figure 1 is a diagram showing the overall configuration of a system 100 for detecting abnormal behavior of users based on logs and filtering false positive data based on feature contributions according to the present invention.

Referring to FIG. 1, a log-based user's abnormal behavior detection and filtering system 100 of false positive data based on feature contribution (hereinafter simply referred to as “system 100”) includes an anomaly detection unit 10 and a filtering unit ( 20).

The anomaly detection unit 10 performs the function of detecting abnormalities in the inspection data, that is, determining whether it is positive or negative, using a pre-learned anomaly detection model 12, and the filtering unit 20 performs the function of determining whether the data for inspection is positive or negative. ) performs a function of classifying false positives or true positives by a pre-learned filtering model (23) for test data determined to be positive and removing false positive test data.

Here, positive means a case in which there is a high possibility of abnormal behavior, and negative means a case in which the behavior is judged to be normal.

Figure 2 is a diagram showing the configuration of the abnormality detection unit 10.

Referring to FIG. 2, the anomaly detection unit 10 may include a preprocessor 11 and an anomaly detection model 12.

The preprocessing unit 11 is a means for performing preprocessing on input data. Here, the input data is log data.

Preprocessing refers to normalization processing to match the range of values for each feature of the input data, missing value processing for missing values present in the data, and one-hot encoding for categorical variables. This means processing, etc.

Here, the input data is log data, which can be, for example, data about the organization's entities (e.g. users, hosts, facilities, IP addresses, and applications), and anomaly detection identifies the possibility of abnormal actions such as leaking internal information. This means identifying high-ranking users.

This input data can be divided into inspection data, learning data, and evaluation data. Inspection data is data that is subject to anomaly detection by the system 100, learning data is data used to train the anomaly detection model 12, and evaluation data is data used to train the filtering model 23. This is the data used.

The anomaly detection model 12 performs learning to detect anomalies based on training data among the input data, and outputs a result of whether the inspection data among the input data is abnormal, that is, positive or negative.

The anomaly detection model 12 may be a machine learning-based neural network model, and is a model based on unsupervised learning that is learned from training data without labels.

To learn the anomaly detection model 12 based on unsupervised learning, the following method can be used.

In other words, the preprocessed data is classified into normal data and abnormal data, and the normal data is divided at a ratio of, for example, about 7:3 and included in the learning data and evaluation data.

Additionally, abnormal data is included only in evaluation data. An anomaly detection model (12) is trained using training data containing only normal data.

Meanwhile, the concepts of machine learning, neural network models, etc. and the method of performing unsupervised learning are known in the prior art, and are not a direct purpose of the present invention, so detailed descriptions are omitted here.

Once learning is completed, the anomaly detection model 12 can output an anomaly detection result, that is, positive or negative, for the inspection data among the input data.

Next, the filtering unit 20 will be described.

Figure 3 is a diagram showing the configuration of the filtering unit 20.

Referring to FIG. 3, the filtering unit 20 includes a labeling generating unit 21, a feature contribution generating unit 22, and a filtering model 23.

The labeling generation unit 21 inputs the evaluation data into the anomaly detection unit 10, and determines true positive (TP) and true positive (TP) for the evaluation data determined to be positive (anomaly) in the anomaly detection unit 10. It is a means of performing labeling with false positives (FP).

As described above, the evaluation data input to the labeling generation unit 21 includes only positive (abnormal) data.

Here, true positive (TP) means that the anomaly detection model 12 of the anomaly detection unit 10 predicted data that was actually positive (anomaly) as positive (anomaly), and false positive (FP) means anomaly detection. This means that model 12 predicted data that was actually negative (normal) as positive (abnormal).

The labeled evaluation data is transmitted to the feature contribution generation unit 22.

The feature contribution generation unit 22 has a function of generating contribution information on the positive prediction of each feature for input data and generating secondary features by replacing each feature with the generated contribution information. Perform.

The input data input to the feature contribution generator 22 is evaluation data judged positive by the anomaly detection unit 10, which has been labeled by the labeling generator 21 when learning the filtering model 23. At the time of detection, it is test data judged positive by the abnormality detection unit 10.

First, when learning the filtering model 23, the feature contribution generation unit 22 selects each feature of the evaluation data determined to be positive in the anomaly detection unit 10 labeled by the labeling generation unit 21 ( feature) is predicted to be positive, and secondary features are created by replacing each feature with the generated contribution information.

In addition, when detecting an anomaly, the feature contribution generation unit 22 generates contribution information for each feature of the test data determined to be positive by the anomaly detection unit 10 in predicting it as positive, and generates information on the contribution of each feature. Secondary features are created by replacing the generated contribution information.

Here, contribution information refers to information indicating the influence of each feature on the result predicted to be positive.

In general, a machine learning-based classification model can be considered a black box model in which the higher the performance, the greater the model's complexity (flexibility) and the lower the interpretability (the degree to which a person can understand the cause of the model's decisions). there is. Models with high complexity show high performance, but there is a problem that it is difficult for people to understand the prediction results, and understanding the prediction leads to trust in the model. Therefore, in the present invention, for the interpretation of model prediction results, (eXplainable Artificial Inteligence) Considers contribution information, which is one of the technologies.

The following methods can be used to generate contribution information.

For example, for each feature used while the input data goes through the anomaly detection model 12, the difference between the model predictions when each feature is included and when it is not included is calculated for all possible subsets of features. By calculating, contribution information for each feature can be generated.

Once contribution information for each feature is generated, secondary features are created by replacing the original feature with the contribution information.

However, this method is illustrative, and other methods that can evaluate the influence of each feature can of course be used.

Figure 4 is a diagram showing an example of secondary features generated by contribution information.

Figure 4 (A) shows the features that the input data originally had, and Figure 4 (B) shows secondary features in which each feature is replaced with contribution information by the feature contribution generator 22.

The generated secondary features are passed to the filtering model (23).

As described above, the filtering model 23 is learned by secondary features generated by the feature contribution generator 22 for the evaluation data, and after learning is completed, the anomaly detection unit 10 determines that the filtering model 23 is positive. It receives the secondary features generated by the feature contribution generation unit 22 for the test data, classifies them as false positives or true positives, and performs a function of removing false positive test data.

The filtering model 23 may also be a machine learning-based neural network model. However, the filtering model 23 is different in that it is supervised learning using secondary features that are replaced with contribution information generated for evaluation data with labels judged positive as described above when performing learning.

When learning by these secondary features is completed, the filtering model 23 classifies the secondary features of the test data determined as positive by the anomaly detection unit 10 as false positive or true positive, and determines the test data to be false positive. By removing the inspection data, the accuracy of abnormality detection can be improved.

Figure 5 shows measurement results evaluating the performance of the system 100 according to the present invention.

In Figure 5, SWaT and KDD are data sets used for learning of the system 100, and before filtering are performance measurement results when the filtering unit 20 is not applied, and after filtering are the performance measurement results when the filtering unit 20 is applied. This is the performance measurement result.

The parts in bold indicate items with improved performance indicators, and it can be seen that the overall performance indicators have improved for each of the two data sets shown.

Since the filtering unit 20 of the system 100 according to the present invention filters false positives (FP) from the prediction results of the anomaly detection unit 10, the recall is the ratio of true positives (TP) to actual positives (anomalies). It can be seen that Precision, which is the ratio of predicted positives (abnormalities) to true positives (TP), changed more significantly when the filtering unit 20 was applied, with little change.

Figure 6 shows a comparison of the ROC-Curve before and after applying the filtering unit 20.

In FIG. 6, ML indicates the anomaly detection unit 10, and ML + Filter indicates the case where the filtering unit 20 is applied to the anomaly detection unit 10.

In both figures, it can be seen that when the filtering unit 20 is applied, it gets closer to the upper left. When the true positive rate (TPR) on the y-axis is the same, the false positive rate (FPR) on the x-axis is smaller when the filter model is applied, so the case where the filtering unit 20 is applied reduces the false positive (FP) rate more effectively. can confirm.

In the above, the present invention has been described with reference to preferred embodiments of the present invention, but these are illustrative and all changes within the equivalent scope understood by the appended claims and drawings are included in the scope of the present invention. It should be noted that

100...Log-based user abnormal behavior detection and filtering system for false positive data based on feature contribution
10...abnormality detection unit
11...Preprocessing part
12...Anomaly detection model
20...filtering unit
21...Labeling generation unit
22...Feature contribution generation unit
23...Filtering model

Claims (8)

A system for detecting abnormal behavior of users based on logs and filtering false positive data based on feature contributions,
An anomaly detection unit that performs the function of determining whether test data is positive or negative based on a pre-learned anomaly detection model; and
A filtering unit that classifies the test data determined to be positive by the abnormality detection unit as false positive or true positive by a pre-learned filtering model and removes the false positive test data.
A log-based user abnormal behavior detection and filtering system for false positive data based on feature contribution.
In claim 1,
The abnormality detection unit,
A preprocessor that performs preprocessing on input data; and
An anomaly detection model in which learning for anomaly detection is performed based on the training data among the input data and outputs a positive or negative result for the inspection data among the input data.
A log-based user's abnormal behavior detection and filtering system for false positive data based on feature contribution, comprising:
In claim 2,
The input data includes inspection data, which is data subject to anomaly detection, training data used to train an anomaly detection model, and evaluation data used to train a filtering model. A system for detecting abnormal behavior and filtering false positive data based on feature contribution.
In claim 3,
Log-based, characterized in that the data for which preprocessing has been completed by the preprocessor is classified into normal data and abnormal data, the normal data is divided and included in the learning data and evaluation data, and the abnormal data is included only in the evaluation data. A system for detecting abnormal user behavior and filtering false positive data based on feature contributions.
In claim 1,
The filtering unit,
a labeling generation unit that inputs evaluation data into an anomaly detection unit, labels the evaluation data determined as positive by the anomaly detection unit as true positive and false positive, and transmits the labeled evaluation data to a feature contribution generation unit;
a feature contribution generator that generates contribution information for predicting each feature as positive for input data, and replaces each feature with the generated contribution information to generate secondary features; and
The evaluation data is learned using the secondary features generated by the feature contribution generation unit, and after learning is completed, the 2 generated by the feature contribution generation unit is used for the inspection data determined to be positive by the anomaly detection unit. A filtering model that receives primary characteristics, classifies false positives or true positives, and removes false positive test data.
A log-based user's abnormal behavior detection and filtering system for false positive data based on feature contribution, comprising:
In claim 5,
The input data input to the feature contribution generation unit is evaluation data labeled by the labeling generation unit when learning a filtering model, and is inspection data judged positive by the anomaly detection unit when detecting an anomaly. A log-based user abnormal behavior detection and filtering system for false positive data based on feature contribution.
In claim 6,
The feature contribution generation unit,
When performing filtering model learning, information on the contribution of each feature of the evaluation data determined to be positive by the anomaly detection unit labeled by the labeling generation unit to being predicted as positive is generated, and the contribution information generated for each feature is generated. Secondary features are generated by replacing information, and when anomaly is detected, contribution information is generated for each feature of the test data determined to be positive in the anomaly detection unit to be predicted as positive, and the contribution generated for each feature is generated. A log-based user's abnormal behavior detection and filtering system for false positive data based on feature contribution, characterized by generating secondary features by replacing them with information.
In claim 7,
The feature contribution information generator calculates the difference between model prediction values when each feature is included and when each feature is not included for each feature used while the input data goes through the anomaly detection model for a subset of all possible features. By doing so, log-based user abnormal behavior detection and filtering of false positive data based on feature contribution, characterized by generating contribution information for each feature and generating secondary features by replacing existing features with the generated contribution information. system.