CN105634781B - Multi-fault data decoupling method and device - Google Patents

Abstract

The invention discloses a multi-fault data decoupling method and device. The method uses the frequent itemset analysis result using the association analysis method for de-rooting and de-correlation in fault data decoupling. and the de-correlated fault data, use the correlation coefficient matrix in the case of single fault data to select the attribution fault for the alarm data. The decoupling method has the characteristics of the correlation analysis method: high accuracy and robustness, and compared with the manual method in the existing network, the work efficiency is improved, and it provides the possibility for large-scale data mining and analysis of fault alarm data.

Description

A method and device for decoupling multi-fault data

technical field

The present invention relates to the field of communication technologies, and in particular, to a method and device for decoupling multiple fault data.

Background technique

A failure in a communication network refers to a malfunction of the hardware or software devices that make up the managed network. An alarm in a communication network refers to an event report composed of a notification sent by a managed object when a specific event occurs, and is used to transmit alarm information. The relationship between faults and faults, and between faults and alarms in a communication network is relatively complex.

Between faults, one fault can exist independently, or one fault may cause multiple faults. For example, a fault on the IUB port causes a user to drop calls, and a power system fault causes a single board to lose power, thereby causing a user to drop calls when the cell is withdrawn. Between faults and alarms, one fault may generate one alarm or multiple alarms. The presence of an alarm also indicates that there may be a failure, not necessarily a failure.

According to the alarm information in the network, the fault analysis of the communication network is one of the important tasks in the research of network maintenance and management. At present, in the research of communication network fault analysis based on data mining method, there have been many research results in the method of fault analysis method for single fault data information according to alarm information through various classification algorithms of data mining.

The data collected in the actual network is the multiple alarm information and the corresponding multiple fault data in the same area and time. There are both co-occurrence of related faults, that is, the coexistence of root-cause faults and dependent faults, and the co-occurrence of multiple unrelated faults.

Therefore, to analyze the root cause of faults according to the alarm information according to the existing research methods, it is necessary to consider the method of decoupling the collected alarm data and the data between multiple faults in the case of multiple faults:

In the case of multiple faults, the correlation analysis between fault data is carried out;

For related multiple faults, give the root cause of the fault;

Gives the attribution fault of the alarm data.

In the actual network, network maintenance engineers are currently used for manual data processing. On the one hand, this method has high labor cost, and the accuracy is limited by the level of engineers. On the other hand, the work efficiency cannot meet the needs of fault analysis of big data.

SUMMARY OF THE INVENTION

The invention provides a multi-fault data decoupling method and device, which are used to solve the problem that the data decoupling method adopted in the prior art is inefficient and cannot meet the fault analysis requirements of big data.

According to an aspect of the present invention, a method for decoupling multiple fault data is provided, comprising:

Obtain K groups of alarm data and K groups of fault data collected in the same area at the same time, where each group of fault data is sorted by fault priority;

Using an association analysis algorithm for the K groups of fault data, a frequent fault item set X is obtained, and the frequent fault itemset X is converted into a fault pairwise correlation matrix R;

Based on the fault pairwise correlation matrix R, perform fault de-correlation and root cause for the fault data groups with multiple faults in the K groups of fault data;

Extract each group of alarm data corresponding to each fault data group with multiple irrelevant faults after fault de-correlation and root cause, and determine each alarm in the extracted groups of alarm data according to the correlation between the alarm and each fault attributable fault.

Optionally, in the method of the present invention, the transformation principle for converting the frequent fault item set X into the fault pairwise correlation matrix R is: mark the pairwise faults that exist simultaneously in any frequent item set as correlation, and for all Pairs of faults that do not exist at the same time in the frequent item set are marked as irrelevant; the elements in the fault pairwise correlation matrix R indicate whether the two faults are correlated.

Optionally, in the method of the present invention, based on the fault pairwise correlation matrix R, the principle of performing fault de-correlation and root cause for the fault data group with multiple faults in the K groups of fault data is as follows:

If the fault pair correlation matrix R indicates that the two related faults exist in the fault data group with multiple faults at the same time, then if the high priority fault exists, the high priority fault is retained, and the low priority fault is deleted;

If two faults that are not correlated in the fault pairwise correlation matrix R exist simultaneously in the fault data group of multiple faults, the two faults are retained at the same time.

Optionally, in the method of the present invention, determining the fault to which each alarm belongs in the extracted groups of alarm data according to the correlation between the alarm and each fault, specifically includes:

For each group of extracted alarm data, obtain each fault contained in the corresponding fault de-correlation and root-caused fault data group, and obtain a fault set;

For each group of extracted alarm data, it is determined that the fault with the highest correlation between each alarm in the alarm data and each fault in the corresponding fault set is the fault to which the corresponding alarm belongs.

Optionally, in the method of the present invention, the method further includes: calculating a Pearson correlation coefficient between each alarm and each fault according to the fault data group of a single fault in the K groups of fault data, and using the Pearson correlation coefficient to represent the alarm Correlation with each fault.

According to another aspect of the present invention, a multi-fault data decoupling device is provided, comprising:

The data input unit is used to obtain K groups of alarm data and K groups of fault data collected in the same area at the same time, wherein each group of fault data is sorted by fault priority;

The data processing unit is used to use an association analysis algorithm on the K groups of fault data to obtain a frequent fault item set X, convert the frequent fault item set X into a fault pairwise correlation matrix R, and based on the fault pairwise correlation matrix R , perform fault de-correlation and root cause for the fault data group with multiple faults in the K group fault data;

The decoupling unit is used to extract each group of alarm data corresponding to each fault data group with multiple irrelevant faults after fault de-correlation and root cause, and determine the extracted groups according to the correlation between the alarm and each fault The fault to which each alarm in the alarm data belongs.

Optionally, in the device of the present invention, the data processing unit converts the frequent fault itemsets X into the fault pairwise correlation matrix R. The conversion principle is: for the pairwise faults that exist at the same time in any frequent item set, the faults are marked as related. , the pairwise faults that do not exist simultaneously in all frequent item sets are marked as irrelevant; the elements in the fault pairwise correlation matrix R indicate whether the pairwise faults are correlated.

Optionally, in the device of the present invention, the data processing unit performs the principle of fault de-correlation and root cause for the fault data groups with multiple faults in the K groups of fault data based on the fault pairwise correlation matrix R. for:

If the fault pair correlation matrix R indicates that the two related faults exist in the fault data group with multiple faults at the same time, then if the high priority fault exists, the high priority fault will be retained and the low priority fault will be deleted; The correlation matrix R indicates that two irrelevant faults exist in the multi-fault fault data group at the same time, then the two faults are retained at the same time.

Optionally, in the device of the present invention, the fault decoupling unit is specifically configured to, for each group of extracted alarm data, obtain the corresponding fault de-correlation and root cause fault data groups included in the group. For each set of extracted alarm data, determine the fault with the highest correlation between each alarm in the alarm data and each fault in the corresponding fault set as the fault to which the corresponding alarm belongs.

Optionally, in the device of the present invention, the data processing unit is further configured to calculate the Pearson correlation coefficient between each alarm and each fault according to the fault data group of a single fault in the K groups of fault data, so as to pass the Pearson correlation coefficient between each alarm and each fault. The correlation coefficient is used to represent the correlation between the alarm and each fault.

The beneficial effects of the present invention are as follows:

The technical scheme disclosed by the invention has the characteristics of the correlation analysis method: high accuracy and strong robustness, and compared with the manual method in the existing network, the work efficiency is improved, and the possibility of large-scale data mining and analysis of fault alarm data is provided. .

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are just some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1 is a flowchart of a method for decoupling multi-fault data according to Embodiment 1 of the present invention;

2 is a flowchart of a method for decoupling multi-fault data according to Embodiment 2 of the present invention;

FIG. 3 is a structural block diagram of a multi-fault data decoupling device provided by the present invention.

Detailed ways

In order to solve the problem that the data decoupling method adopted in the prior art is inefficient and cannot meet the fault analysis requirements of big data, the present invention provides a multi-fault data decoupling method and device. The innovation of the solution provided by the present invention lies in that the frequent itemset analysis results using the association analysis method are used for de-rooting and de-correlation in fault data decoupling, and based on the de-causing and de-correlated fault data, use The correlation coefficient matrix in the case of single fault data selects the attribution fault for the alarm data. The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1

An embodiment of the present invention provides a multi-fault data decoupling method, as shown in FIG. 1 , including the following steps:

Step S101, acquiring K groups of alarm data and K groups of fault data collected in the same area at the same time, wherein each group of fault data is sorted by fault priority;

Wherein, each group of alarm data includes M data, and each data corresponds to an alarm to indicate whether the alarm exists;

Each set of fault data contains N pieces of data, and each data corresponds to a fault to indicate whether the fault exists.

Step S102, use an association analysis algorithm on the K groups of fault data to obtain a frequent fault item set X, and convert the frequent fault item set X into a fault pairwise correlation matrix R;

Among them, the transformation principle of transforming the fault frequent itemset X into the fault pairwise correlation matrix R is: for the pairwise faults that exist at the same time in any frequent item set, it is marked as correlation, and for all frequent item sets, there is no pairwise fault that exists at the same time. marked as irrelevant;

Elements in the fault pairwise correlation matrix R indicate whether the pairwise faults are correlated.

Step S103, based on the fault pairwise correlation matrix R, perform fault de-correlation and root cause on the fault data groups with multiple faults in the K groups of fault data;

Wherein, based on the fault pairwise correlation matrix R, the principle of performing fault de-correlation and root cause for the fault data group with multiple faults in the K group fault data is as follows:

If the fault pair correlation matrix R indicates that the two related faults exist in the fault data group with multiple faults at the same time, then if the high priority fault exists, the high priority fault is retained, and the low priority fault is deleted;

If two faults that are not correlated in the fault pairwise correlation matrix R exist simultaneously in the fault data group of multiple faults, the two faults are retained at the same time.

Step S104, extract each group of alarm data corresponding to each fault data group with multiple irrelevant faults after fault de-correlation and root cause, and determine the alarm data in each group of extracted alarm data according to the correlation between the alarm and each fault. The fault to which each alarm belongs.

Among them, according to the correlation between the alarm and each fault, the fault to which each alarm belongs in the extracted sets of alarm data is determined, which specifically includes:

For each group of extracted alarm data, obtain each fault contained in the corresponding fault de-correlation and root-caused fault data group, and obtain a fault set;

For each group of extracted alarm data, it is determined that the fault with the highest correlation between each alarm in the alarm data and each fault in the corresponding fault set is the fault to which the corresponding alarm belongs.

The correlation between the alarm and the fault is preferably represented by a Pearson correlation coefficient.

The calculation method of the Pearson correlation coefficient is as follows: according to the fault data group of a single fault in the K groups of fault data, the Pearson correlation coefficient between each alarm and each fault is calculated. The specific calculation method involved is a known technology, and will not be described in detail here.

To sum up, the multi-fault data decoupling scheme described in this embodiment has the characteristics of high accuracy and strong robustness, and compared with the manual method in the existing network, the work efficiency is improved, and the large-scale fault alarm data is improved. Data mining analysis provides the possibility.

Embodiment 2

This embodiment provides a multi-fault data decoupling method. The implementation principle of the method is the same as that of the first embodiment. By disclosing more technical details for implementing the method of the present invention, the specific implementation of the present invention is more clearly expressed process. It should be noted that this embodiment is a preferred embodiment, and the disclosed content is not used to exclusively limit the implementation process of the present invention.

This embodiment provides a method for decoupling fault data in the case of multiple faults in a communication network, as shown in FIG. 2 , including the following steps:

Step 1: Data collection and preprocessing method:

For communication networks, define the priority of failures and sort them by priority. The priority of the fault can be evaluated according to the number of network elements affected by the fault, the number of hardware, and the criticality of the KPI (Key Performance Indicator, key performance indicator) affected by the fault.

The faults sorted by priority (in order to be distinguished from subsequent fault data, described below by fault variables) are denoted as {G ₁ , G ₂ , . . . , G _N }. For example, taking the network element NODEB as an example, the set of fault variables can be: {NODEB is powered off..NODEB is out of service, NODEB control board is faulty..IUB is disconnected..}

The system alarm (in order to be distinguished from the subsequent alarm data, the following is expressed by the alarm variable) is recorded as {E ₁ , E ₂ , . . . , E _M }. For example, {NODEB power failure alarm, ..RRU out of service, inter-board communication traffic exceeds the alarm threshold, performance threshold is out of bounds}.

Collect K groups of alarm data and prioritized K groups of fault data in the live network to form the following matrix:

Among them, the element e _im (1<=i<=K, 1<= _m <=M) in the matrix records whether the alarm variable Em exists in the i- _th group of sampling data: if the alarm variable Em exists, then e _im =1, otherwise e _im =0.

Among them, the element g _in (1<=i<=K, 1<=n<=N) in the matrix records whether the fault variable G _n exists in the i-th group of sampling data: if the alarm variable G _n exists, then g _in = 1, otherwise g _in = 0.

Assuming that there are multiple faults in the i-th sampled data, then there are multiple non-zero items in g _i1 ... g _iN , such as: g _i1 ... g _iN = {1,0,...1,0.. }

Step 2: Use the Apriori association analysis algorithm to obtain frequent itemsets X for the K groups of fault information samples. Assuming that the number of frequent itemsets obtained is J, record the frequent itemsets of fault information as {x ₁ , x ₂ ,...,x _J }, where x ₁ ～x _J are all fault variables {G ₁ ,G ₂ ,...,G _N } is a subset of the set. For example, x _j ={NODEB is powered off, NODEB is taken out of service}, where j=1, . . . , J.

Step 3: Transform the fault frequent itemset X into the fault pairwise correlation matrix R.

The element r _xy in the fault pairwise correlation matrix R is defined as the pairwise correlation coefficient of faults _Gx and _Gy . The calculation method of r _xy is as follows: if no G _x and G _y exist at the same time in all frequent item sets, then r _xy =0, otherwise r _xy =1. Among them, x=1,...,N; Y=1,...,N

Step 4: According to the fault pairwise correlation matrix R, perform fault de-correlation and root cause on the multi-fault data set in the sample.

For the i-th group of fault data, if there are multiple non-zero items in g _i1 ... g _iN , it is considered as multi-fault data, then the fault data group g _i1 ... g _iN is de-correlated and root cause The transformation operation is transformed into the fault data set g′ _i1 ... g′ _iN after decorrelation and root cause. Among them, the calculation method of g′ _in (n=1,...,N) is as follows:

g' _in = g _in , if g' _in is non-zero, then:

Search in all faults g _i1 , g _i2 ,...g _i(n-1) whose priority is higher than the current fault, if there is a certain fault data g _in' non-zero, and the fault is related to the fault of the current fault Coefficient _rn'n =1, then let _g'in =0.

Step 5: Filter the single-fault data group, and calculate the alarm variables {E ₁ ,E ₂ ,...,E _M } and the fault variables {G ₁ ,G ₂ ,...,G according to the single-fault data group Pearson's correlation coefficient between _N }. Define the Pearson correlation coefficient between the alarm _Em and the fault Gn as _{p mn} .

Step 6: Traverse the fault data after fault de-correlation and root cause. If a certain fault data {g′ _i1 ... g′ _iN } is multi-irrelevant fault data, analyze the corresponding multi-irrelevant fault data Each alarm in the alarm data {e _i1 ... e _iM } belongs to the fault.

For the i-th group of sampled data, if there are multiple non-zero items in g′ _i1 ... g′ _iN , it is considered as multi-irrelevant fault data.

If e _im is non-zero (that is, there is an alarm), the method for analyzing the fault attributable to e _im is as follows:

The faults corresponding to the non-zero items in {g′ _i1 ... g′ _iN } are formed into a fault set, and the fault with the largest Pearson correlation coefficient with the alarm E _m in the fault set is found as the attributable fault of e _im .

Embodiment 3

An embodiment of the present invention provides a multi-fault data decoupling device. Each unit involved in the device can be implemented by hardware plus a software program. The software program is used to realize the functions of the following units. The hardware uses It is used to provide support for the operation of software programs, thereby forming a physical hardware device. As shown in FIG. 3 , the device in this embodiment includes:

A data input unit 310, configured to acquire K groups of alarm data and K groups of fault data collected in the same area at the same time, wherein each group of fault data is sorted by fault priority;

The data processing unit 320 is configured to use an association analysis algorithm on the K groups of fault data to obtain a frequent fault item set X, convert the frequent fault itemset X into a fault pairwise correlation matrix R, and based on the fault pairwise correlation matrix R, perform fault de-correlation and root cause for the fault data group with multiple faults in the K group of fault data;

The decoupling unit 330 is configured to extract each group of alarm data corresponding to each fault data group with multiple irrelevant faults after fault de-correlation and root cause, and determine the extracted alarm data according to the correlation between the alarm and each fault. The fault to which each alarm in the group alarm data belongs.

Based on the above-mentioned structural framework and implementation principles, several specific and preferred implementations under the above-mentioned structure are given below to refine and optimize the functions of the device of the present invention, specifically involving the following contents:

In this embodiment, the transformation principle for the data processing unit 320 to convert the fault frequent item set X into the fault pairwise correlation matrix R is: for the pairwise faults that exist simultaneously in any frequent item set to be marked as correlation, for all frequent item sets, Pairwise faults that do not exist at the same time are marked as irrelevant; the elements in the fault pairwise correlation matrix R indicate whether the pairwise faults are correlated.

In the present embodiment, the data processing unit 330 performs fault de-correlation and root cause for the fault data groups with multiple faults in the K groups of fault data based on the fault pairwise correlation matrix R as follows:

If the fault pair correlation matrix R indicates that the two related faults exist in the fault data group with multiple faults at the same time, then if the high priority fault exists, the high priority fault will be retained and the low priority fault will be deleted; The correlation matrix R indicates that two irrelevant faults exist in the multi-fault fault data group at the same time, then the two faults are retained at the same time.

In this embodiment, the fault decoupling unit 330 is specifically configured to, for each group of extracted alarm data, obtain each fault contained in the corresponding fault-decorrelated and root-caused fault data group, and obtain a fault set; For each group of extracted alarm data, it is determined that the fault with the highest correlation between each alarm in the alarm data and each fault in the corresponding fault set is the fault to which the corresponding alarm belongs.

Preferably, in this embodiment, the data processing unit 320 is further configured to calculate the Pearson correlation coefficient between each alarm and each fault according to the fault data group of a single fault in the K groups of fault data, so as to use the Pearson correlation coefficient to calculate the Pearson correlation coefficient. Indicates the correlation between an alarm and each fault.

The multi-fault data decoupling scheme described in this embodiment has the characteristics of high accuracy and strong robustness, and improves the work efficiency compared with the manual method in the existing network, and provides a large-scale data mining analysis of fault alarm data. possible.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (8)

1. a multi-fault data decoupling method, is characterized in that, comprises: Obtain K groups of alarm data and K groups of fault data collected in the same area at the same time, where each group of fault data is sorted by fault priority; Using an association analysis algorithm for the K groups of fault data, a frequent fault item set X is obtained, and the frequent fault itemset X is converted into a fault pairwise correlation matrix R; Based on the fault pairwise correlation matrix R, perform fault de-correlation and root cause for the fault data groups with multiple faults in the K groups of fault data; Extract each group of alarm data corresponding to each fault data group with multiple irrelevant faults after fault de-correlation and root cause, and determine each alarm in the extracted groups of alarm data according to the correlation between the alarm and each fault attributable fault; Among them, the transformation principle of transforming the fault frequent item set X into the fault pairwise correlation matrix R is: for the pairwise faults that exist simultaneously in any frequent item set, it is marked as correlation, and for all frequent item sets, there is no simultaneous two pairs of faults. The two faults are marked as irrelevant; Elements in the fault pairwise correlation matrix R indicate whether the pairwise faults are correlated. 2. The method according to claim 1, characterized in that, based on the fault pairwise correlation matrix R, performing fault de-correlation and root cause on the fault data groups with multiple faults in the K groups of fault data. The principle is: If the fault pair correlation matrix R indicates that the two related faults exist in the fault data group with multiple faults at the same time, then if the high priority fault exists, the high priority fault is retained, and the low priority fault is deleted; If two faults that are not correlated in the fault pairwise correlation matrix R exist simultaneously in the fault data group of multiple faults, the two faults are retained at the same time. 3. The method according to claim 1, wherein determining the fault to which each alarm belongs in the extracted alarm data of each group according to the correlation between the alarm and each fault, specifically comprising: For each group of extracted alarm data, obtain each fault contained in the corresponding fault de-correlation and root-caused fault data group, and obtain a fault set; For each group of extracted alarm data, it is determined that the fault with the highest correlation between each alarm in the alarm data and each fault in the corresponding fault set is the fault to which the corresponding alarm belongs. 4. The method according to claim 1 or 3, wherein in the method, according to the fault data group of a single fault in the K groups of fault data, the Pearson correlation coefficient between each alarm and each fault is calculated, and The correlation between alarms and faults is represented by the Pearson correlation coefficient. 5. A multi-fault data decoupling device, comprising: The data input unit is used to obtain K groups of alarm data and K groups of fault data collected in the same area at the same time, wherein each group of fault data is sorted by fault priority; The data processing unit is used to use an association analysis algorithm on the K groups of fault data to obtain a frequent fault item set X, convert the frequent fault item set X into a fault pairwise correlation matrix R, and based on the fault pairwise correlation matrix R , perform fault de-correlation and root cause for the fault data group with multiple faults in the K group fault data; The decoupling unit is used to extract each group of alarm data corresponding to each fault data group with multiple irrelevant faults after fault de-correlation and root cause, and determine the extracted groups according to the correlation between the alarm and each fault The fault to which each alarm in the alarm data belongs; Wherein, the transformation principle for the data processing unit to convert the fault frequent item set X into the fault pairwise correlation matrix R is as follows: for the pairwise faults that exist simultaneously in any frequent item set, it is marked as correlation, and for all frequent item sets that do not have simultaneous faults at the same time Existing pairwise faults are marked as irrelevant; the elements in the fault pairwise correlation matrix R indicate whether the pairwise faults are correlated. 6 . The device according to claim 5 , wherein the data processing unit performs fault de-correlation and rooting on the fault data groups with multiple faults in the K groups of fault data based on the fault pairwise correlation matrix R. 7 . The principle of factorization is: If the fault pair correlation matrix R indicates that the two related faults exist in the fault data group with multiple faults at the same time, then if the high priority fault exists, the high priority fault will be retained and the low priority fault will be deleted; The correlation matrix R indicates that two irrelevant faults exist in the multi-fault fault data group at the same time, then the two faults are retained at the same time. 7 . The device according to claim 5 , wherein the fault decoupling unit is specifically configured to obtain, for each group of extracted alarm data, a corresponding fault data group after de-correlation and root cause. 8 . Each fault contained in the fault set is obtained; for each group of extracted alarm data, the fault with the highest correlation between each alarm in the alarm data and each fault in the corresponding fault set is determined as the fault to which the corresponding alarm belongs. 8. The apparatus according to claim 5 or 7, wherein the data processing unit is further configured to calculate the Pearson difference between each alarm and each fault according to the fault data group of a single fault in the K groups of fault data The correlation coefficient is used to express the correlation between the alarm and each fault through the Pearson correlation coefficient.

Images