CN118944958A - Message rule matching method and device and electronic equipment - Google Patents

Abstract

The application discloses a message rule matching method and device and electronic equipment. Wherein the method comprises the following steps: acquiring a valid message at a network outlet; extracting feature data of the effective message; determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data, and splicing the metadata fields; and carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result comprises retrieval rule attributes corresponding to the effective message. The application solves the technical problems of insufficient recognition precision and poor flexibility caused by rough rule matching for message detection in the related technology.

Description

Message rule matching method and device and electronic equipment

Technical Field

The present application relates to the field of computer communications, and in particular, to a method and an apparatus for matching a message rule, and an electronic device.

Background

Deep Packet Inspection (DPI) is an advanced network monitoring strategy that improves the traditional way of inspecting only the communication protocol header, going deep into the application layer's detailed analysis and screening of each packet's specific content. This approach extends the capability of traffic monitoring to enable full insight and manipulation of the transmission details at each place in the network. The DPI technology plays a key role in guaranteeing network safety, realizing content filtering, effectively managing network traffic and the like. However, this technique has a high dependency on hardware performance because of the large number of data parsing tasks to be handled. With the widespread adoption of broadband networks, handling larger scale traffic makes the demand for high performance hardware more stringent.

In addition, the types of application programs in the current network environment are increasing, each application has a unique detection specification, so that a larger rule set and higher operation requirement are generated. To address these issues, DPI systems are often equipped with high performance processors and agile software designs that can quickly adjust and adapt to new threats or changing application scenarios. Constructing and maintaining an efficient DPI solution is not just a process of technical deployment, but also needs to be constantly optimized to ensure high-speed operation and accurate analysis results. Such systems must have the flexibility to handle ever-increasing data load capabilities and accommodate diverse network applications. However, in the related art, rule matching for message detection is rough, and the problems of insufficient recognition accuracy and poor flexibility exist.

In view of the above problems, no effective solution has been proposed at present.

Disclosure of Invention

The embodiment of the application provides a method and a device for matching message rules and electronic equipment, which at least solve the technical problems of insufficient recognition precision and poor flexibility in the prior art that the rule matching for message detection is rough.

According to an aspect of the embodiment of the present application, there is provided a method for matching a message rule, including: acquiring a valid message at a network outlet; extracting feature data of the effective message; determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data, and splicing the metadata fields; and carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result comprises retrieval rule attributes corresponding to the effective message.

Optionally, after obtaining the valid message at the network outlet, the method further includes: obtaining quintuple information of an effective message, wherein the quintuple information comprises a source IP, a destination IP, a source address, a destination address and a transport layer protocol; determining a hash value corresponding to the effective message according to the quintuple information; and determining an output path of the effective message according to the hash value.

Optionally, before extracting the feature data of the valid message, the method further includes: judging whether the effective message exists in the flow table record according to the five-tuple information of the effective message to obtain a first judging result; under the condition that the first judging result indicates that no effective message exists in the flow table records, a record is newly established for storing the effective message, and a UNKNOWN label is allocated to the newly established flow table record corresponding to the effective message; judging whether the label of the message record is UNKNOWN or not under the condition that the first judging result indicates that the message record corresponding to the effective message exists in the flow table record, and obtaining a second judging result; if the second judgment result is negative, no valid message is recognized; if the second judgment result is yes or a record is newly established to store the effective message, identifying a packet header protocol stack and load data of the effective message; and determining five-tuple information, a packet header protocol stack and load data of the effective message as characteristic data of the effective message.

Optionally, determining a metadata field with a filtering rule in a packet header protocol stack of the valid packet from the feature data includes: determining the protocol type of the effective message from the packet header protocol stack; determining whether the transport layer where each protocol type is located carries load data; and under the condition that the transport layer where each protocol type is located carries load data, analyzing the corresponding protocol to determine a metadata field.

Optionally, rule matching is performed on the spliced metadata field and a preset field to obtain a matching result, which includes: acquiring a payload in a valid message; obtaining an initial matching result according to the effective load, the spliced metadata field and the preset field, wherein the initial matching result is used for representing a matching result of the secondary rule unit; and determining a matching result according to the initial matching result, wherein the matching result is used for representing the matching result of a primary rule unit, and the primary rule unit is a parent rule unit of a secondary rule unit.

Optionally, obtaining an initial matching result according to the payload, the spliced metadata field and the preset field, including: determining the payload as a first lookup parent string and the spliced metadata field as a second lookup parent string; determining a first searching substring used for carrying out load matching from a preset field, and determining a second searching substring used for carrying out metadata matching; and matching the first search parent string with the first search child string, and matching the second search parent string with the second search child string to obtain an initial matching result.

Optionally, determining the matching result according to the initial matching result includes: determining the id number of the secondary rule unit from the initial matching result; determining all parent rule units corresponding to the id numbers of the secondary rule units from the multi-level rule unit storage table; the parent rule element is determined to be a matching result.

According to another aspect of the embodiment of the present application, there is also provided a device for matching a message rule, including: the acquisition module is used for acquiring the effective message at the network outlet; the extraction module is used for extracting the characteristic data of the effective message; the determining module is used for determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data and splicing the metadata fields; and the matching module is used for carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result comprises a retrieval rule attribute corresponding to the effective message.

According to still another aspect of the embodiment of the present application, there is also provided an electronic device including: a memory for storing program instructions; a processor coupled to the memory for executing program instructions that perform the following functions: acquiring a valid message at a network outlet; extracting feature data of the effective message; determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data, and splicing the metadata fields; and carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result comprises retrieval rule attributes corresponding to the effective message.

According to still another aspect of the embodiments of the present application, there is further provided a nonvolatile storage medium, where the nonvolatile storage medium includes a stored computer program, and a device where the nonvolatile storage medium is located executes the matching method of the foregoing packet rule by running the computer program.

According to still another aspect of the embodiments of the present application, there is also provided a computer program product including computer instructions which, when executed by a processor, implement the above-mentioned method of matching message rules.

In the embodiment of the application, the effective message at the network outlet is obtained; extracting feature data of the effective message; determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data, and splicing the metadata fields; and carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result contains the retrieval rule attribute corresponding to the effective message, so that the purpose of detecting the refined message is achieved, the technical effect of providing high message identification precision is realized, and the technical problems of rough rule matching, insufficient identification precision and poor flexibility of message detection in the related technology are solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:

FIG. 1 is a block diagram of a hardware architecture of a computer terminal for implementing a matching method for message rules according to an embodiment of the present application;

FIG. 2 is a flow chart of a method of matching message rules according to an embodiment of the application;

FIG. 3 is a process flow diagram of a protocol metadata extraction unit according to an embodiment of the present application;

Fig. 4 is a block diagram of a message rule matching device according to an embodiment of the present application.

Detailed Description

In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, partial terms or terminology appearing in the course of explaining the embodiments of the present application are applicable to the following explanation:

DPI (DEEP PACKET Inspection ): the deep packet inspection technology is to add application protocol identification of application layer data, data packet content inspection and deep decoding to the traditional IP data packet inspection technology (inspection and analysis of data packet elements contained between OSI L2-L4).

The DPI in the related art has the following limitations, such as too rough rule matching, limited coverage, difficulty in supporting complex regular expressions and rule combinations, and thus has the problems of insufficient recognition accuracy, poor flexibility in query and the like. In order to solve the problems in the related art, the embodiment of the present application provides a method for matching a message rule, which may be run in a computer terminal shown in fig. 1, and the computer terminal is explained below.

The embodiment of the method for matching the message rule provided by the embodiment of the application can be executed in a mobile terminal, a computer terminal or similar computing devices. Fig. 1 shows a block diagram of the hardware architecture of a computer terminal for implementing a matching method of message rules. As shown in fig. 1, the computer terminal 10 may include one or more processors (shown as 102a, 102b, … …,102n in the figures) which may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and/or wireless network. In addition, the method may further include: a display, a keyboard, a cursor control device, an input/output interface (I/O interface), a Universal Serial BUS (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a BUS. It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 1 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

It should be noted that the one or more processors and/or other data processing circuits described above may be referred to herein generally as "data processing circuits. The data processing circuit may be embodied in whole or in part in software, hardware, firmware, or any other combination. Furthermore, the data processing circuitry may be a single stand-alone processing module or incorporated, in whole or in part, into any of the other elements in the computer terminal 10. As referred to in embodiments of the application, the data processing circuit acts as a processor control (e.g., selection of the path of the variable resistor termination connected to the interface).

The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the matching method of the message rule in the embodiment of the present application, and the processor executes the software programs and modules stored in the memory 104, thereby executing various functional applications and data processing, that is, implementing the matching method of the message rule. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission module 106 is used to receive or transmit data via a network. The specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission module 106 may be a Radio Frequency (RF) module for communicating with the internet wirelessly.

The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 10.

It should be noted here that, in some alternative embodiments, the computer terminal shown in fig. 1 may include hardware elements (including circuits), software elements (including computer code stored on a computer readable medium), or a combination of both hardware and software elements. It should be noted that fig. 1 is only one example of a specific example, and is intended to illustrate the types of components that may be present in the computer terminals described above.

In the above-described operating environment, embodiments of the present application provide an embodiment of a method for matching message rules, and it should be noted that the steps illustrated in the flowchart of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order other than that illustrated herein.

Fig. 2 is a flowchart of a method for matching message rules according to an embodiment of the present application, as shown in fig. 2, the method includes the following steps:

Step S202, obtaining effective message at network outlet.

In the above step S202, the valid packet refers to a packet that is correctly processed and forwarded at the network egress, usually has a correct destination address and port number, conforms to the network protocol standard, and can be correctly routed and transmitted in the network. Acquiring valid messages at the network outlet may help a network administrator monitor network traffic and detect potential security issues. By analyzing the effective message, an administrator can know the communication condition in the network, discover abnormal activities in time and take necessary measures to protect the network security.

In the embodiment of the present application, the step S202 may be implemented by a data acquisition unit, where the data acquisition unit may be deployed in a network in a serial or bypass manner, receive packet data in the network, and screen an effective portion from the packet data, and transfer the effective portion to a subsequent flow. The valid message is a message having a potential recognition value in a scene such as application recognition, terminal recognition, or user recognition in DPI recognition. In one session, only a necessary part of messages are identified, unnecessary identification of the subsequent messages is omitted under the condition that the preamble messages are identified, the overall performance of the DPI can be greatly improved, and the processing pressure of the subsequent units is reduced.

Step S204, extracting the characteristic data of the effective message.

In the step S204, extracting the feature data of the effective message refers to acquiring and analyzing the key attribute and information of the effective message from the network communication. These characteristic data may include source address, destination address, port number, protocol type, transmission rate, data size, time stamp, etc. By extracting the characteristic data of the effective message, a network administrator can be helped to better know the mode and trend of the network flow, identify abnormal behaviors in the network, and perform network performance optimization and security monitoring.

In the embodiment of the present application, step S204 may be implemented, for example, by a message processing unit, where the message processing unit includes two subunits, that is, a message parsing unit and a stream association processing unit, respectively. The main function of the message parsing Unit (MESSAGE PARSING Unit, MPU) is to receive and process data messages, and analyze and decode the received original data according to a specific protocol format. Its specific responsibilities include: frame delimitation, verification, splitting, reassembly, etc. The flow association processing unit is a component specifically designed to understand and analyze connectivity and patterns in network traffic. This unit will keep track of the various connections in the TCP/IP protocol stack, understand how each packet belongs to a particular session or flow, and extract key information from it, such as source IP, destination IP, port number, and content of the application layer, etc. The stream association processing unit also performs a certain degree of data association, but does not associate the packets themselves, but correlates the content extracted from each packet in a stream. In many cases, the content extracted by the preamble can help skip unnecessary identification of the current and subsequent frames, so as to save a lot of performance resources.

Step S206, determining the metadata field with filtering rule in the packet header protocol stack of the effective message from the characteristic data, and splicing the metadata field.

In the above step S206, in the network communication, each data packet contains metadata fields related to information about its source and destination, transmission protocol, and the like. These metadata fields may include information of source IP address, destination IP address, source port number, destination port number, protocol type, etc. The network administrator can determine the packet header protocol stack of the valid packet according to the metadata fields, and formulate the corresponding filtering rules to filter and process the data packet. Splicing metadata fields refers to grouping together these metadata fields in a format for easier analysis and processing. For example, the source IP address and the destination IP address may be stitched together, separated by a slash; the source port number and the destination port number are spliced together, separated by a colon, etc. By splicing the metadata fields, a unified data format can be formed, and subsequent data processing and analysis are facilitated.

In the embodiment of the present application, the step S206 may be implemented by a protocol metadata extraction unit, which is a core component of the deep packet inspection system, and has a main task of extracting important information about a communication protocol from a network packet. In addition, the metadata of the protocol needs to be spliced in the session to form a metadata pattern string for matching with the subsequent character string. The splicing method of the protocol metadata follows three principles:

1. Only the protocol metadata fields covered in the rule set of the system are spliced, and the protocol metadata field values which do not appear in the rules are not spliced into the mode string;

2. The metadata pattern string may be spliced, for example, in a manner of "& & metadata field= metadata field value". If the rule contains the integer metadata field with multiple values in comparison of the size relation, only splicing the metadata field, and not splicing the field value;

3. The splicing order of the metadata mode strings is spliced according to the protocol analysis order, and all metadata fields are in parallel relation in the mode strings due to the fear of the splicing mode, so that the splicing order does not influence the matching result.

In the above principle, the first principle ensures that only valid fields contained in a rule are spliced, and fields that are not present in the rule are not hit by any rule. Many nonsensical copying and scanning processes are omitted. In the second principle, the size relation comparison and integer data with multiple values are included, and as the equal relation does not exist, whether the rule hits or not can not be directly determined through pattern string matching, and only the metadata fields are spliced in the pattern string, and after other sub-elements of the rule hit, the size relation of the integer metadata is compared. If other sub-elements miss, the integer metadata field need not be compared. The main function of the metadata mode string is to identify the metadata information of the message in the session and match the metadata attribute in the preset rule, so that a plurality of metadata screening conditions can be added in the identification rule, and the range of the DPI filtering rule is greatly expanded.

Step S208, rule matching is carried out on the spliced metadata fields and preset fields, and a matching result is obtained, wherein the matching result contains retrieval rule attributes corresponding to the effective messages.

In step S208, rule matching is performed on the spliced metadata field and the preset field, which means that the metadata field in the network data packet is compared with a rule set in advance to determine whether the data packet meets the rule requirement. In the matching process, the data packet can be compared with the rule according to the definition and the condition of the rule to obtain a matching result. The matching result contains the retrieval rule attribute corresponding to the effective message, namely the characteristics or attributes of the data packet conforming to the rule.

In the embodiment of the present application, the step S208 may be implemented, for example, by a string depth matching unit (hereinafter referred to simply as a matching unit) and an identification rule checking unit (hereinafter referred to simply as a checking unit), where the matching unit is a module specifically used for searching and analyzing a specific string or pattern string in the data packet. The matching unit includes the following functions: pattern construction, data packet traversal, metadata splicing character string traversal, depth matching algorithm, complexity processing, result processing and other functions. The verification unit is a component for verifying whether the content of the data packet accords with a preset series of rules or strategies, and is different from the matching unit, and the verification unit only integrates and verifies the matching results of the units, because the filtering conditions supported by the traditional DPI rule are single, or the filtering conditions are strong by a single packet, and the information combination among multiple data packets is not supported. Therefore, an identification rule checking unit is added to connect the matching results of different data packets in the session in series and perform comprehensive checking to give a final identification result. Compared with the processing of the message and the pattern string matching, the checking process does not consume much performance, but can greatly expand the possibility of DPI rule functionally, and the traditional five-tuple rule, the load rule, the regular filtering condition and the metadata field, even the dynamic filtering rule obtained by converting the content analyzed and extracted from the data packet, can be arbitrarily combined to meet the increasingly growing DPI filtering requirement.

The attributes defining the rule include "protocol number: four-element group: flow feature elements: protocol metadata: regular elements: load feature element, processing action after hit. The matching and verification of the rules are also in the sequence of protocol number- > tetrad- > protocol metadata- > stream feature element- > regular element- > load feature element ". The matching cost of each element attribute is also complied, the rule element with the minimum cost is subjected to the preferential matching test, and the subsequent rule element is not required to be matched again if the preamble element is not hit, so that the great performance cost can be saved. The protocol number, the quadruple and the stream feature element are relatively common, and are further described with respect to protocol metadata. Protocol metadata can be defined for any field of any protocol, and protocol metadata can be matched across data packets in a stream, although DPI processing does not correlate messages in the stream at the stream level, protocol analysis on each packet in the stream can summarize single-packet protocol metadata in the stream, such as an upstream request and a downstream response of HTTP in a stream, and at this time, an HTTP request and an HTTP response are defined in a rule, accurate filtering can be performed even if the request and the response occur in data packets with different upstream and downstream directions, and the HTTP example aims to illustrate that the rule has the capability of matching metadata across packets on the premise of DPI.

In the embodiment of the present application, an actual DPI system may include millions of rules, and the following describes, by way of example, the attribute of the rule and the management manner of the corresponding rule element with reference to two rules:

rule 1: "TCP,: 80-: * Flow.pktnum= = 43, http.url= = "helloworld", http host= "fuwuqiyuming", target list ". This is a target list filtering rule that filters 80 port TCP flow packets equal to 43, defining the HTTP protocol field.

Rule 2: "TCP,: 443-: * Flow. Pktnum > = 40, ssl. Sni= = "fuwuqiyuming", target list ". This is a target list filtering rule that filters 443 port TCP flow packets greater than 40, defining the ssl protocol field.

And integrating the two rules, wherein the protocol number rule is integrated into one protocol number rule, namely TCP. This means that the preamble unit will only perform rule-related processing for TCP data, and that non-TCP data will be ignored. The five-tuple element only comprises bidirectional port rules, two five-tuple rules are integrated after the two rules are integrated, namely 80 port rules and 443 port rules, and all non-target ports are ignored. The flow characteristic rule defining the number of packets appears in both rules and this field may contain a comparison of the size relationship. Therefore, the rule is integrated as flow pktnum > =40. All streams with packet numbers less than 40 frames are not matched with the following protocol metadata, stream characteristics, regular elements and load characteristic elements. Such integration aims at filtering streams with a packet number of less than 40 frames, but does not change the packet number matching rule of rule one. The rule will also screen the stream with a packet number of 43 among the streams with a packet number greater than 40 frames. The protocol metadata contains a composite rule of host and url of HTTP, the rule only screens request frames possibly containing HostUrl uplink data packets for tcp flows of 80 ports, and rule 2 is the same. Defining such rule attributes enables filtering attributes with large filtering strength and small performance overhead to function first, so that meaningless metadata, flow characteristics, regular elements and load characteristic filtering are avoided, and the elements are the main overhead of performance in DPI engineering.

In addition, in practical DPI engineering, data can hit multiple rules, and in fact, granularity between rules is unequal, and when the results are integrated after the hit rules, the results with more accurate preset conditions and finer recognition levels should be preferentially output. This depends on the fact that in the checking unit, a definition of the priority of the rule is preset, the recognition result with high output priority should be preferentially considered when the plurality of filtering rules are hit, and the parallel output recognition result is considered when there is no conflict between the recognition results and the priority is the same.

By virtue of the design of the identification rule checking unit, a rule multi-stage searching method can be provided to improve the condition of inflexibility in DPI rule inquiry, namely, complete rules in the DPI system can be searched in real time through any unit in the rule combination.

In the embodiment of the present application, the main storage structure of the identification rule checking unit may be a multi-level rule unit storage hash table, and the primary rule is stored in a rule array, where each primary rule unit contains all elements of the rule: rule id, elements of all secondary rules, and index for each secondary rule. Each subordinate secondary rule unit is stored in a different secondary storage unit according to its rule type, e.g. the five-tuple rule is stored in a different five-tuple hash according to the corresponding hash. Because any type of five-tuple filtering method is supported, according to the attribute of the actual rule, there are actually multiple five-tuple storage arrays of different types, for example, a rule storage table with the source direction ip as an index, a rule storage table with the source direction ip mask as an index, and the like, and although the search main keys of the table entries are different, the search principle is similar to that of the storage structure, so that only any one of the table entries needs to be understood, and the other table entries are similar. In the actual matching process, the object hit by the rule element is a second-level rule unit, in order to reversely find the father-level rule unit when the second-level rule is hit, the mapping from the second-level rule unit to the first-level rule unit needs to be recorded, and because different first-level rule units possibly contain the same second-level rule unit, the mapping from one second-level rule unit to the first-level rule unit is one-to-many and is stored in the second-level rule unit in the form of a first-level rule id array to serve as a search index, and as long as one second-level rule unit is hit, all the possibly hit first-level rule units can be found through the id index, so that the overall matching performance is greatly improved.

Through the steps S202 to S208, the purpose of detecting the refined message is achieved, so that the technical effect of providing high message identification precision is achieved, and the technical problems of insufficient identification precision and poor flexibility caused by rough rule matching for message detection in the related technology are solved. The following description is given.

In step S202 in the above method for matching a packet rule, after obtaining a valid packet at a network egress, the method further includes: obtaining quintuple information of an effective message, wherein the quintuple information comprises a source IP, a destination IP, a source address, a destination address and a transport layer protocol; determining a hash value corresponding to the effective message according to the quintuple information; and determining an output path of the effective message according to the hash value.

In the embodiment of the application, the quintuple information refers to a source IP address, a destination IP address, a source port number, a destination port number and a transport layer protocol in a network data packet. By extracting these five-tuple information, a data packet can be uniquely identified, and important information such as source and destination addresses, protocol types, etc. of its communication can be determined. The hash value is a unique identifier calculated from the contents of the data packet. By hashing the five-tuple information of the valid message, a unique hash value may be generated to represent the packet. The hash value may be used to determine the routing path of the data packet, i.e. to determine to which node or output path in the network the valid message should be sent. According to a pre-defined hash algorithm and routing rules, the next transmission path of the effective message can be determined according to the hash value. Through the steps, the data flow in the network can be managed and controlled more effectively, the transmission path of the data packet is determined according to the quintuple information and the hash value, and the accuracy and the safety of network communication are ensured. The methods help to improve network performance and security, and ensure that valid messages can be transmitted and processed according to the correct path.

In step S204 in the above method for matching a message rule, before extracting the feature data of the valid message, the method further includes: judging whether the effective message exists in the flow table record according to the five-tuple information of the effective message to obtain a first judging result; under the condition that the first judging result indicates that no effective message exists in the flow table records, a record is newly established for storing the effective message, and a UNKNOWN label is allocated to the newly established flow table record corresponding to the effective message; judging whether the label of the message record is UNKNOWN or not under the condition that the first judging result indicates that the message record corresponding to the effective message exists in the flow table record, and obtaining a second judging result; if the second judgment result is negative, no valid message is recognized; if the second judgment result is yes or a record is newly established to store the effective message, identifying a packet header protocol stack and load data of the effective message; and determining five-tuple information, a packet header protocol stack and load data of the effective message as characteristic data of the effective message.

In step S206 in the above method for matching a packet rule, determining, from the feature data, a metadata field with a filtering rule in a packet header protocol stack of an effective packet, where the metadata field includes: determining the protocol type of the effective message from the packet header protocol stack; determining whether the transport layer where each protocol type is located carries load data; and under the condition that the transport layer where each protocol type is located carries load data, analyzing the corresponding protocol to determine a metadata field.

In the embodiment of the present application, the protocol type of the valid message, for example TCP, UDP, IP, is determined according to the information in the feature data. According to the protocol type, it is determined whether the protocol carries payload data in the transport layer. For example, TCP and UDP protocols typically carry payload data, whereas IP protocols do not. For protocol types that carry payload data, such as TCP and UDP, it is necessary to further parse the protocol to determine metadata fields therein, such as source port number, destination port number, sequence number, acknowledgement number, etc. These metadata fields can be used for further analysis and processing of the valid messages. Through the steps, the metadata field with the filtering rule in the packet header protocol stack of the effective message can be accurately determined from the characteristic data, wherein the metadata field comprises the protocol type and the load data condition, and the corresponding protocol is analyzed to acquire more detailed metadata field information.

In step S208 of the foregoing method for matching a message rule, rule matching is performed on the spliced metadata field and a preset field, so as to obtain a matching result, including: acquiring a payload in a valid message; obtaining an initial matching result according to the effective load, the spliced metadata field and the preset field, wherein the initial matching result is used for representing a matching result of the secondary rule unit; and determining a matching result according to the initial matching result, wherein the matching result is used for representing the matching result of a primary rule unit, and the primary rule unit is a parent rule unit of a secondary rule unit.

In the embodiment of the application, the effective load is extracted from the effective message, namely the data content carried by the effective message. The payload generally refers to the actual data transmitted in the data packet, not the header information. And matching the payload with the spliced metadata field and the preset field to obtain an initial matching result. This initial match result is used to represent the match of the secondary rule element, i.e. the specific application of the rule is determined from the match of the payload and metadata fields. And further determining a matching result according to the initial matching result, wherein the matching result is used for representing the matching condition of the primary rule unit. The primary rule unit refers to a parent rule unit of the secondary rule unit, namely, the matching condition of the whole rule is determined based on the initial matching result.

In the above step, according to the payload, the spliced metadata field and the preset field, an initial matching result is obtained, including: determining the payload as a first lookup parent string and the spliced metadata field as a second lookup parent string; determining a first searching substring used for carrying out load matching from a preset field, and determining a second searching substring used for carrying out metadata matching; and matching the first search parent string with the first search child string, and matching the second search parent string with the second search child string to obtain an initial matching result.

In the embodiment of the application, the payload is used as a first search parent string, namely the main body data which needs to be matched. And meanwhile, taking the spliced metadata field as a second lookup parent string, namely metadata information which needs to be matched with the effective load. A first search substring for matching with the payload is determined in a preset field, which is a key or pattern for matching the payload. At the same time, a second search substring is determined for matching the metadata field, which is a keyword or pattern for matching the metadata field. The initial matching result can be obtained by matching the first search parent string with the first search child string and matching the second search parent string with the second search child string. These results will reflect the matching of the payload and metadata fields in the preset fields, thereby determining the application of the rules and the matching results.

In the above step, determining a matching result according to the initial matching result includes: determining the id number of the secondary rule unit from the initial matching result; determining all parent rule units corresponding to the id numbers of the secondary rule units from the multi-level rule unit storage table; the parent rule element is determined to be a matching result.

In the embodiment of the application, the ID number of the secondary rule unit, namely the unique identifier representing the rule unit, is determined according to the initial matching result. And searching all parent rule units corresponding to the ID numbers in the multi-level rule unit storage table according to the ID numbers of the two-level rule units. These parent rule units include a primary rule unit, a higher-level rule unit of a secondary rule unit, and other related rule units, etc. And taking all the parent rule units determined from the multi-level rule unit storage table as final matching results. These parent rule elements will aggregate the results of the matching of the payload and metadata fields, as well as the application of the rules. And determining a matching result according to the initial matching result, wherein the matching result comprises the steps of determining the ID number of the corresponding rule unit, searching all relevant parent rule units, and taking the parent rule units as a final matching result. The hierarchical rule matching and result determining method is helpful for more comprehensively knowing the matching condition of the rules, and meanwhile, the rule management and application are more flexible and efficient.

In order to better understand the matching method of the message rule provided by the embodiment of the application, the following explanation is given to the above steps:

Step one: the data acquisition unit firstly utilizes convergence streaming equipment deployed in an outlet area or a core area of network traffic to converge traffic in a plurality of optical fibers, filters retransmission data packets, error packets of failure requests and other data packets which are not generated artificially and are caused by network delay, and then indicates stream hash corresponding to an ith frame message according to hash values hashi =CRC (srcipi, srcporti, dstipi, dstporti, protoi) of the remaining data packets. And finally, forwarding all the data packets to I/O interfaces of a plurality of parallel message processing units through a load balancing technology according to the hash value and the number of the parallel message processing units.

Step two: the message processing unit determines whether a corresponding record exists in the flow table ST＝{(srcip1,srcport1,dstip1,dstport1,proto1)：label1,..,(srcipτ,srcportτ,dstipτ,dstportτ,protoτ)：labelτ} according to the five-tuple of the data message, if the corresponding record exists and labeli is not UNKNOWN, the flow label is directly allocated, the identification result in the flow is inherited, and the current message is not processed. If the corresponding record in the flow table does not exist, a flow table node is newly established to allocate a UNKNOWN flow label for the flow table node, and subsequent processing is carried out. If the corresponding record in the flow table already exists, the flow label is UNKNOWN, and the backward processing is continued. After frame delimitation, verification, splitting, recombination and other processes, five-tuple information, application layer load and packet protocol stack extracted from the message are stored in a stream record. The five-tuple rule in the follow-up identification rule is matched, the original data source matched by the protocol rule is the corresponding flow record of the flow table in the message processing unit. And constructing the association relation between the messages through the flow table record.

Step three: the protocol metadata extraction unit is used for extracting the packet header protocol stack according to the packet header protocol stack acquired by the message processing unit, for example, "Ethernet: IP: UDP: GPRS: IP: TCP: HTTP ", and the header start position of each layer protocol, extracts necessary higher layer protocol data. Meanwhile, the protocol metadata extraction unit processes according to the type of the transmission layer, existence of a data load and other conditions, and takes the transmission layer effective load of part of the message as the input of the subsequent character string depth matching unit. Finally, the protocol metadata obtained by the unit is effectively spliced, for example, #http.host= "www.texaslotto.com" & http.request.method= "GET".

FIG. 3 is a flowchart of a protocol metadata extraction unit according to an embodiment of the present application, where, as shown in FIG. 3, the protocol metadata extraction unit performs classification processing according to a transport layer protocol, if the protocol is a TCP stream higher layer protocol, predicts and identifies the higher layer protocol according to stream information, determines whether the TCP carries a load, if not, does not parse, ends the flow, if carries a load, then polls and identifies the higher layer protocol, interprets the corresponding higher layer protocol, and splices metadata fields of the higher layer protocol; if the UDP flow is processed by the high-level protocol, judging whether the UDP carries a load, if the UDP does not carry the load, the analysis is not carried out, the process is ended, if the UDP carries the load, the high-level protocol is identified by polling, the corresponding high-level protocol is interpreted, and the metadata fields of the high-level protocol are spliced; if the other protocol higher layer protocol is processed, the higher layer protocol is predicted and identified according to the stream information, the higher layer protocol is identified in a polling way, the corresponding higher layer protocol is interpreted, and the metadata fields of the higher layer protocol are spliced.

Step four: the character string depth matching unit is responsible for taking the effective load in the data packet and the spliced protocol metadata character string as a searching parent string, and taking preset keywords, character strings, wildcards, regular expressions and the like as searching substrings to carry out efficient identification matching. To achieve such functionality, specially designed libraries or tools, such as libpcap (for grabbing network traffic), pypcap, or scapy (Python libraries for packet parsing and construction) may be used. The specific code implementation depends on the language and technology stack used, but the core logic typically includes reading the data packets, decoding the application layer data, and then performing string matching operations. In practical DPI systems, this matching tends to be highly optimized because of the large amount of real-time network traffic that needs to be handled. Therefore, efficient algorithms and data structures (such as suffix arrays, trie trees, or Aho-Corasick automata) are preferred for implementation, and intermediate matching implementations will not be described in detail since the design of the present application is not limited to specific detailed implementations. After the load matching and the metadata matching, the depth matching unit returns a corresponding matching result, and the result is presented in the form of id numbers of each secondary rule unit, for example, "id load=1, 16, id protocol=1, 8", and is output to the identification rule checking unit.

Step five: the recognition rule checking unit searches all the possible primary rule units according to the hit condition of the secondary rule units, and if any missed secondary rule unit exists under a certain primary rule, the missed primary rule unit can be judged without examining other secondary rule units, so that part of unnecessary rule searching process is omitted. For example, the attribute of a certain rule is #http.host= "www.texaslotto.com" & http.request.method= "POST", and the attribute of the rule analyzed in the actual message is #http.host= "www.texaslotto.com" & http.request.method= "GET", when the two-level field of http.request.method is found to be unmatched in the process of comparing, all other two-level rule attributes under the first-level rule do not need to be compared and recorded. After screening all the hit second-level rule units through the above process, obtaining all the hit first-level rule units.

Fig. 4 is a structural diagram of a message rule matching device according to an embodiment of the present application, as shown in fig. 4, the device includes:

an obtaining module 40, configured to obtain a valid message at the network egress;

An extracting module 42, configured to extract feature data of the valid message;

A determining module 44, configured to determine metadata fields with filtering rules in a packet header protocol stack of the valid packet from the feature data, and splice the metadata fields;

and the matching module 46 is configured to perform rule matching on the spliced metadata field and a preset field to obtain a matching result, where the matching result includes a retrieval rule attribute corresponding to the valid message.

In the obtaining module of the message rule matching device, the obtaining module is further configured to obtain five-tuple information of the valid message, where the five-tuple information includes a source IP, a destination IP, a source address, a destination address, and a transport layer protocol; determining a hash value corresponding to the effective message according to the quintuple information; and determining an output path of the effective message according to the hash value.

In the extracting module of the matching device of the message rule, the extracting module is further configured to determine whether an effective message exists in the flow table record according to five-tuple information of the effective message, so as to obtain a first determination result; under the condition that the first judging result indicates that no effective message exists in the flow table records, a record is newly established for storing the effective message, and a UNKNOWN label is allocated to the newly established flow table record corresponding to the effective message; judging whether the label of the message record is UNKNOWN or not under the condition that the first judging result indicates that the message record corresponding to the effective message exists in the flow table record, and obtaining a second judging result; if the second judgment result is negative, no valid message is recognized; if the second judgment result is yes or a record is newly established to store the effective message, identifying a packet header protocol stack and load data of the effective message; and determining five-tuple information, a packet header protocol stack and load data of the effective message as characteristic data of the effective message.

In the determining module of the matching device of the message rule, the determining module is further used for determining the protocol type of the effective message from the packet header protocol stack; determining whether the transport layer where each protocol type is located carries load data; and under the condition that the transport layer where each protocol type is located carries load data, analyzing the corresponding protocol to determine a metadata field.

In the matching module of the matching device of the message rule, the matching module is also used for acquiring the effective load in the effective message; obtaining an initial matching result according to the effective load, the spliced metadata field and the preset field, wherein the initial matching result is used for representing a matching result of the secondary rule unit; and determining a matching result according to the initial matching result, wherein the matching result is used for representing the matching result of a primary rule unit, and the primary rule unit is a parent rule unit of a secondary rule unit.

In the matching module of the matching device of the message rule, the matching module is further configured to determine the payload as a first lookup parent string, and determine the spliced metadata field as a second lookup parent string; determining a first searching substring used for carrying out load matching from a preset field, and determining a second searching substring used for carrying out metadata matching; and matching the first search parent string with the first search child string, and matching the second search parent string with the second search child string to obtain an initial matching result.

In the matching module of the matching device of the message rule, the matching module is also used for determining the id number of the secondary rule unit from the initial matching result; determining all parent rule units corresponding to the id numbers of the secondary rule units from the multi-level rule unit storage table; the parent rule element is determined to be a matching result.

It should be noted that, the message rule matching device shown in fig. 4 is used for executing the message rule matching method shown in fig. 2, so the explanation of the message rule matching method is also applicable to the message rule matching device, and will not be repeated here.

The embodiment of the application also provides electronic equipment, which comprises a memory and a processor, wherein the memory is used for storing program instructions; the processor is coupled to the memory for executing program instructions that perform the following functions: acquiring a valid message at a network outlet; extracting feature data of the effective message; determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data, and splicing the metadata fields; and carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result comprises retrieval rule attributes corresponding to the effective message.

It should be noted that, the above electronic device is configured to execute the matching method of the message rule shown in fig. 2, so that the explanation related to the matching method of the message rule is also applicable to the electronic device, which is not described herein.

The embodiment of the application also provides a nonvolatile storage medium, which comprises a stored computer program, wherein the equipment of the nonvolatile storage medium executes the following matching method of the message rule by running the computer program: acquiring a valid message at a network outlet; extracting feature data of the effective message; determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data, and splicing the metadata fields; and carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result comprises retrieval rule attributes corresponding to the effective message.

It should be noted that, the above-mentioned nonvolatile storage medium is used to execute the matching method of the message rule shown in fig. 2, so the explanation related to the matching method of the message rule is also applicable to the nonvolatile storage medium, and will not be repeated here.

The embodiments of the present application also provide a computer program product comprising computer instructions which, when executed by a processor, implement the steps of the matching method of message rules in the various embodiments of the present application.

The embodiment of the application also provides a computer program which realizes the steps of the message rule matching method in the various embodiments of the application when being executed by a processor.

The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely a preferred embodiment of the present application and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present application, which are intended to be comprehended within the scope of the present application.

Claims (11)

1. The message rule matching method is characterized by comprising the following steps:

acquiring a valid message at a network outlet;

Extracting characteristic data of the effective message;

Determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data, and splicing the metadata fields;

and carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result comprises retrieval rule attributes corresponding to the effective message.

2. The method of claim 1, wherein after obtaining the valid message at the network egress, the method further comprises:

obtaining quintuple information of the effective message, wherein the quintuple information comprises a source IP, a destination IP, a source address, a destination address and a transport layer protocol;

determining a hash value corresponding to the effective message according to the five-tuple information;

and determining the output path of the effective message according to the hash value.

3. The method of claim 2, wherein prior to extracting the characteristic data of the valid message, the method further comprises:

judging whether the effective message exists in the flow table record according to the five-tuple information of the effective message to obtain a first judging result;

under the condition that the first judging result indicates that the effective message does not exist in the flow table record, a record is newly established for storing the effective message, and a UNKNOWN label is distributed for the newly established flow table record corresponding to the effective message;

Judging whether the label of the message record is UNKNOWN or not under the condition that the first judging result indicates that the message record corresponding to the effective message exists in the flow table record, and obtaining a second judging result;

If the second judgment result is negative, the valid message is not recognized any more;

If the second judgment result is yes or a record is newly created to store the effective message, identifying a packet header protocol stack and load data of the effective message; and determining the five-tuple information of the effective message, the packet header protocol stack and the load data as characteristic data of the effective message.

4. The method of claim 1, wherein determining from the feature data a metadata field in a header protocol stack of the valid message having a filtering rule, comprises:

determining the protocol type of the effective message from the packet header protocol stack;

Determining whether the transport layer where each protocol type is located carries load data;

and under the condition that the transport layer where each protocol type is located carries load data, analyzing the corresponding protocol to determine a metadata field.

5. The method of claim 1, wherein performing rule matching on the spliced metadata field and the preset field to obtain a matching result comprises:

acquiring a payload in the effective message;

Obtaining an initial matching result according to the payload, the spliced metadata field and the preset field, wherein the initial matching result is used for representing a matching result of a secondary rule unit;

And determining the matching result according to the initial matching result, wherein the matching result is used for representing the matching result of a primary rule unit, and the primary rule unit is a parent rule unit of the secondary rule unit.

6. The method of claim 5, wherein obtaining an initial matching result based on the payload, the spliced metadata field, and the preset field comprises:

Determining the payload as a first lookup parent string and the spliced metadata field as a second lookup parent string;

determining a first searching substring used for carrying out load matching from the preset field, and determining a second searching substring used for carrying out metadata matching;

and matching the first search parent string with the first search child string, and matching the second search parent string with the second search child string to obtain the initial matching result.

7. The method of claim 5, wherein determining the match result based on the initial match result comprises:

determining the id number of a secondary rule unit from the initial matching result;

Determining all father level rule units corresponding to the id numbers of the two-level rule units from a multi-level rule unit storage table;

And determining the parent rule unit as the matching result.

8. A message rule matching device, comprising:

The acquisition module is used for acquiring the effective message at the network outlet;

the extraction module is used for extracting the characteristic data of the effective message;

The determining module is used for determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data and splicing the metadata fields;

And the matching module is used for carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result comprises the retrieval rule attribute corresponding to the effective message.

9. An electronic device, comprising:

A memory for storing program instructions;

A processor, coupled to the memory, for executing program instructions that perform the following functions: acquiring a valid message at a network outlet; extracting characteristic data of the effective message; determining metadata fields with filtering rules in a packet header protocol stack of the effective message from the characteristic data, and splicing the metadata fields; and carrying out rule matching on the spliced metadata fields and preset fields to obtain a matching result, wherein the matching result comprises retrieval rule attributes corresponding to the effective message.

10. A non-volatile storage medium, wherein the non-volatile storage medium comprises a stored computer program, and the device in which the non-volatile storage medium is located executes the matching method of the message rule according to any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising computer instructions which, when executed by a processor, implement a method of matching message rules as claimed in any one of claims 1 to 7.