CN104734916B - A kind of high-efficiency multi-stage anomalous traffic detection method based on Transmission Control Protocol - Google Patents

Abstract

The invention requests protection of a high-efficiency multi-level abnormal flow detection method based on the TCP protocol, adding a multi-level abnormal flow detection mechanism to the traditional abnormal flow detection process. This method is to detect the abnormality of the data flow sent by the client in the network. It uses the differential mean method to perform differential stabilization on the original flow generated by the client, and at the same time analyzes and counts according to the existing flow in the network. Set an adaptive threshold interval, perform adaptive threshold differential traffic detection on the stabilized traffic, and perform further anomaly detection on the data packets that pass the primary detection. This further anomaly detection is mainly to analyze the data packet forwarded by the route, extract its key field, and further judge whether the data packet sent from the client is abnormal according to the judgment of the key field. The invention improves detection accuracy and is simple and easy to realize.

Description

An Efficient Multi-level Abnormal Traffic Detection Method Based on TCP Protocol

technical field

The invention belongs to the technical field of communication anomaly detection, relates to fast and real-time anomaly detection technology for various anomalies on the Internet, and specifically designs a high-efficiency multi-level abnormal traffic detection method based on the TCP protocol.

Background technique

Network abnormal traffic detection is an important part of network monitoring. Abnormal network traffic refers to the situation where the traffic behavior in the network deviates from the normal behavior. In the network, there are many reasons for abnormal network traffic. For example, equipment failure in the network leads to abnormal communication and abnormality; abnormal network operation, sudden access (Flash crowd), network intrusion, etc. will cause network abnormality. At the same time, network anomaly detection is an important guarantee for communication security during the continuous development of the network, the planning of the network topology is becoming more and more complex, the network equipment is becoming more and more diverse, and the scale of network users is increasing. While network users are looking for convenient communication and trust in the network, new types of network threats are also increasing. How to discover and eliminate these network threats is an important task of network anomaly detection and an important part of ensuring normal network communication.

The attacks and threats faced by the network mainly come from the inside of the network. For example, a large number of network viruses, active attacks of hosts in the network and sudden increase of abnormal network traffic will cause overloading of network equipment, resulting in network congestion and may further lead to network paralysis. . A SYN Flood DDoS attack is an attack that a bad network user takes advantage of the defect in the three-way handshake connection of the TCP protocol to forge the IP address of a normal user, thereby causing immeasurable losses to the network. Therefore, when there are abnormalities in the network, the first measure is to find out these abnormalities and generate abnormal alarms. At the same time, network anomalies will not only attack a certain place, but will spread to the surrounding as wide as possible. Its ultimate purpose is to spread to the largest extent of the network, resulting in multiple types of anomalies. Aiming at this situation, a real-time and fast detection method for finding abnormalities is needed, and the abnormalities are found and truncated, so that the network can communicate normally.

Abnormal traffic on the network is characterized by sudden changes in traffic, and the characteristics of the precursor are unknown, which can bring great harm to the network or computers on the network in a short period of time. Therefore, the abnormal behavior of network traffic can be detected in real time and quickly, and the cause Making a reasonable response is one of the prerequisites to ensure the effective operation of the network, and reducing the losses caused by malicious network attacks is another important aspect of ensuring network security.

At present, the anomaly detection methods that have been proposed, such as the nonlinear abnormal flow detection method (NLPP), the abnormal flow detection method based on wavelet analysis, and the abnormal flow detection method based on the ARMA model, can detect the abnormality quickly in real time, but the computational The complexity is high, and the detection results are not accurate enough, and there is often a large false alarm rate, and the detection method can only be used when the traffic data has long-term correlation characteristics. However, when most flow data are collected, the correlation characteristics are not obvious, and the fluctuation trend often presents a non-stationary state, which makes the scope of use of the detection method very limited. The anomaly detection method proposed by the present invention preprocesses the flow data before the flow detection, so as to overcome the problem of detection limitation. At the same time, on the basis of traditional detection methods, the proposed multi-level detection mechanism effectively reduces the false alarm rate of detection.

Contents of the invention

In view of the deficiencies in the prior art, the purpose of the present invention is to provide a method that ensures the correct rate while reducing the false alarm rate, the method is simple and easy to implement, the technical solution of the present invention is as follows: a high-efficiency multi-stage TCP protocol-based Abnormal traffic detection method, which includes the following steps:

101. Collect network traffic data within a time period T, and then for the original traffic sequence R in the network traffic data, the observed value at time t is represented by x _t , x _t ∈ R, t=1,2,...,T, Remove the unusable flow data value x _t according to |x _t |>kvar_R criterion, where k represents the Grubbs criterion coefficient, var_R represents the variance of the sequence R, and the retained flow data is used as a flow observation sequence X;

102. Perform differential stabilization preprocessing on the flow observation sequence X, and the differential flow sequence obtained by preprocessing is D, where the difference value d _t = x _t -x _t-1 , t>1, d _t ∈ D, t=1 ,2,...N, after obtaining the differential flow sequence D, input it in step 103;

103. Calculate the average value and variance of sequence X and sequence D respectively, and estimate the interval [l _t , h _t ] of the differential flow value at time t according to the average value and variance, Among them, p _t represents the threshold predicted value at time t, l _t and h _t represent the minimum and maximum values of differential flow allowed at time t, respectively, and var_d _t represents the variance of differential flow at time t, which is detected in step 102 After the differential traffic sequence D is input, the firewall starts the primary detection and defense function, and detects the transmitted data according to the threshold prediction value p _t at time t. When the differential traffic data value at time t is within the range of the differential traffic prediction value [l When it is within the range of [l _t , h _t ], it is judged to be normal traffic, and it is forwarded to the server; when it exceeds the range of [l _t , h _t ], it is judged to be abnormal traffic, and jumps to step 104;

104. The multi-level detection system of the firewall decomposes the forwarded data packet, extracts the key field key_field in the data packet, and judges these key fields key_field, and forwards it to the server if no abnormal field is found; If an abnormal field is detected, the packet is discarded;

105. After re-detection in step 104, the normal data packet is forwarded to the server, so that the server and the client establish a handshake connection for the first time;

106. After the first handshake connection is established, the server will send the reply message M _response to the client, and at the same time wait for the confirmation message ACK from the client. When the client receives the reply message M _response from the server, the two ends establish the first The second handshake connection; when the server receives the confirmation message ACK, the server and the client establish a third handshake connection, and the two can communicate.

Further, the steps of differential smoothing preprocessing described in step 102 are:

S21. For the network communication data sequence {x ₁ ,x ₂ ,…,x _T } in the collected time period T, analyze and remove abnormal values and keep normal values. Here, if |x _t |>kvar_R, it means x _i Abnormal value, where k represents the Grubbs criterion coefficient, and var_R represents the variance of the sequence R; the reserved observation sequence is used as the observation sequence X;

S22: Calculate the average value of the original sequence R and its variance var_R;

S23: Perform differential preprocessing on the observation sequence X, d _t ∈ D, t=1,2,...T, where d _t =x _t -x _t-1 , t>1;

S24: For the difference sequence D, calculate its average value and variance var_D.

Further, the average value of the original sequence R The formula is: var_R represents their variance,

Advantage of the present invention and beneficial effect are as follows:

The invention uses a high-efficiency multi-stage abnormal flow detection method based on the TCP protocol to perform multiple detections and judgments on the flow data. Because the existing anomaly detection technology has a high false alarm rate during the detection process, the detection accuracy is greatly affected. In order to ensure the accuracy of detection and the security of network communication. In this paper, we propose a method for multi-level anomaly detection. First, based on offline statistics, the adaptive threshold method is used to estimate the traffic interval at the next moment; then, according to the estimated traffic interval, online detection and judgment are performed; when the judgment result is abnormal, multiple In the process of multi-level detection, the key information in the data packet is extracted and judged. This multi-level detection mechanism effectively reduces the false positive rate of anomaly detection. The present invention adopts a high-efficiency multi-level abnormal flow detection method based on the TCP protocol, and uses the method of differential mean variance to obtain a differential flow with a stable trend. The calculation complexity of the method is low, and offline calculation and online detection are used. The method makes the detection speed fast and can meet the requirements of real-time detection. At the same time, the added multi-level detection mechanism not only meets the requirements of reducing the false alarm rate, but also ensures the security of network communication.

Description of drawings

Fig. 1 is a schematic diagram of a multi-level anomaly detection process of the present invention;

Fig. 2 is a schematic diagram of flow data screening in the present invention;

Fig. 3 is a schematic diagram of sequence stabilization of the present invention;

Fig. 4 is a schematic diagram of the adaptive threshold calculation of the present invention;

Fig. 5 is a schematic diagram of the multi-level anomaly detection of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Apparently, the described embodiment is only one embodiment of the present invention, not all of them.

FIG. 1 is a schematic diagram of the multi-level anomaly detection process of the present invention. The present invention is proposed based on the TCP three-way handshake connection protocol of Internet communication. During the communication process, the client makes a request, requests access to the server, and establishes a connection. When the access is normal, the access process is as follows:

S1: The client initiates a request, and the sent request packet is forwarded to the firewall through the router. After receiving the request packet, the firewall collects statistics on the traffic and performs primary detection, that is, differential traffic detection; if the detected traffic is in the differential traffic prediction In the value range [l _t , h _t ], the firewall will determine that the data packet is a normal data packet and forward it to the server; otherwise, the data packet will be judged as an abnormal data packet; at this time, the firewall will open Further detection; in further abnormal detection, analyze the data packet, extract the key field key_field, and further detect the key_field. At this time, if the detection result is still abnormal, it will be discarded; if it is normal, it will be marked as Misjudgment, and forward it to the server; at this time, the client and the server will establish the first handshake connection;

S2: When the server establishes the first connection with the client, it sends a reply message M _response to the client, and waits for the confirmation message ACK from the client, and at the same time starts the waiting timer T _wait , when the client receives the reply message M _response , the second handshake connection is established;

S3: When the waiting time of the server exceeds the maximum waiting time, the server will discard the data packet; otherwise, when the server receives the confirmation message ACK sent by the client, the third handshake connection is established. At this point, both parties can communicate.

Fig. 2 is a schematic diagram of flow data screening in the present invention. Collect flow data for a period of time T, and the sampling data interval is 1s, then the collected original flow sequence is denoted by R, and the observed value at time t is denoted by x _t , where x _t ∈ R, t=1,2,…, T; use Represents the average value of the original sequence {x ₁ ,x ₂ ,…,x _T }, var_R represents their variance,

Before filtering the traffic data, the observation sequence Determine each value in the sequence R in turn. When the value x t at time _t satisfies |x _t |>kvar_R, remove x _t , where k represents the Grubbs criterion coefficient, otherwise the observation sequence X=X∪ x _t , where x _t ∈ X,t=1,2,...,N.

Fig. 3 is a schematic diagram of sequence stabilization in the present invention. In order to make the differential sequence of the original sequence better reflect the trend of data fluctuations, the differential sequence of the original sequence is defined as D, that is, to preprocess the observation sequence {x ₁ ,x ₂ ,…,x _N }, that is, use d _t Indicates the difference value at time t, d _t =x _t -x _t-1 , t>1, d _t ∈ D, t=1,2,...N. use Denotes the average value of the difference sequence D, and has Then the difference mean at time t is Using var_D to represent the variance of the difference sequence D, there is When N→∞, there is It can be concluded that the differential flow tends to be stable.

Fig. 4 is a schematic diagram of the adaptive threshold calculation of the present invention. When performing online real-time anomaly detection, it is first necessary to calculate the adaptive threshold. When determining the adaptive threshold, use l _t and h _t to denote the minimum and maximum values of differential traffic allowed at time t, respectively. The present invention calculates the adaptive threshold through the refresh mechanism, which is mainly as follows:

The threshold prediction value p _t at time t is obtained by superimposing the differential flow at the previous time, Among them, α represents a weighting constant, which is mainly determined according to the number of hosts sending contracts in the model, that is, to control the proportion of new data in the model, and to control the speed at which the model adapts to local behaviors, thus establishing a normal model refresh mechanism. If the current observed value completely conforms to the normal model, then the observed value at this time is considered normal. However, since the actual situation is difficult to conform to the theoretical model, a confidence interval is set with the standard deviation of the observed value, and the level of the tolerance range is different according to the number of standard deviations added. is 2 to 3 times the standard deviation. Since the data has been differentially preprocessed, this paper uses 2 times the standard deviation for judgment, so the threshold range obtained is where n represents the number of clients. Thus, the adaptive threshold interval is [l _t , h _t ].

Fig. 5 is a schematic diagram of the multi-level anomaly detection of the present invention. The request data packet sent by the client is forwarded to the firewall through the router. After receiving the data packet, the firewall calculates the adaptive threshold interval according to the existing observation sequence. The main calculation method is to first perform the first adaptive threshold detection. In this detection, the traffic data at the next moment is detected mainly according to the statistical data of the network traffic. If it is determined that the data packet is normal during the detection, it is directly forwarded to the server; if it is determined to be abnormal, multi-level detection is performed to further confirm whether the data packet is an abnormal packet.

In further anomaly detection, the data packet is analyzed and the key field key_field is extracted. Further check the key_field, if it is normal, it will be marked as a misjudgment and forwarded to the server; if the test result is still abnormal, it will be discarded. Simultaneously process the next packet in queue Q.

The above embodiments should be understood as only for illustrating the present invention but not for limiting the protection scope of the present invention. After reading the contents of the present invention, skilled persons can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims (3)

1. an efficient multi-level abnormal traffic detection method based on TCP protocol, is characterized in that, comprises the following steps: 101. Collect network traffic data within the time period T, and then for the original sequence R in the network traffic data, the observed value at time t is represented by x _t , x _t ∈ R, t=1,2,...,T, according to |x _t |＞kvar_R criterion to remove unavailable flow data value x _t , where k represents the Grubbs criterion coefficient, var_R represents the variance of the original sequence R, and the retained flow data is regarded as an observation sequence X; 102. Perform difference stabilization preprocessing on the observation sequence X, and the difference sequence obtained by preprocessing is D, where the difference value d _t = x _t -x _t-1 , t>1, d _t ∈ D, t=1,2 ,...N, after obtaining the differential sequence D, input it in step 103; 103. Calculate the average value and variance of the observation sequence X and the difference sequence D respectively, and estimate the interval [l _t , h _t ] of the differential flow value at time t according to the average value and variance, n represents the number of clients, where p _t represents the threshold prediction value at time t, l _t and h _t represent the minimum and maximum values of differential traffic allowed at time t, respectively, var_d _t represents the variance of differential traffic at time t, After detecting the input of the differential sequence D in step 102, the firewall starts the primary detection and defense function, and detects the transmitted data according to the threshold prediction value p _t at time t. When the predicted value is within the range [l _t , h _t ], it is judged to be normal traffic and forwarded to the server; when it exceeds the range [l _t , h _t ], it is judged to be abnormal traffic and skips to step 104; 104. The multi-level detection system of the firewall decomposes the forwarded data packet, extracts the key field key_field in the data packet, and judges these key fields key_field, and forwards it to the server if no abnormal field is found; If an abnormal field is detected, the packet is discarded; 105. After re-detection in step 104, the normal data packet is forwarded to the server, so that the server and the client establish a handshake connection for the first time; 106. After the first handshake connection is established, the server will send the reply message M _response to the client, and at the same time wait for the confirmation message ACK from the client. When the client receives the reply message M _response from the server, the two ends establish the first The second handshake connection; when the server receives the confirmation message ACK, the server and the client establish a third handshake connection, and the two can communicate. 2. the efficient multi-stage abnormal flow detection method based on TCP protocol according to claim 1, characterized in that, the step of differential stabilization preprocessing described in step 102 is: S21. For the network communication data sequence {x ₁ , x ₂ ,…, x _T } within the collected time period T, analyze and remove abnormal values and keep normal values. Here, if |x _t |＞kvar_R means x _t Abnormal value, where k represents the Grubbs criterion coefficient, var_R represents the variance of the original sequence R; the reserved observation sequence is used as the observation sequence X; S22: Calculate the average value of the original sequence R and its variance var_R; S23: Perform differential preprocessing on the observation sequence X, there are d _t ∈ D, t=1,2,...T, where d _t =x _t -x _t-1 , t>1; S24: For the difference sequence D, calculate its average value and variance var_D. 3. the efficient multi-stage abnormal traffic detection method based on TCP protocol according to claim 2, is characterized in that, the average value of original sequence R The formula is: var_R represents the variance of the original sequence,