CN115209452B - Method and device for checking hidden danger of core network, electronic equipment and storage medium - Google Patents
Method and device for checking hidden danger of core network, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115209452B CN115209452B CN202110387834.2A CN202110387834A CN115209452B CN 115209452 B CN115209452 B CN 115209452B CN 202110387834 A CN202110387834 A CN 202110387834A CN 115209452 B CN115209452 B CN 115209452B
- Authority
- CN
- China
- Prior art keywords
- data
- alarm
- module
- preprocessing
- judged
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000003860 storage Methods 0.000 title claims abstract description 14
- 230000005856 abnormality Effects 0.000 claims abstract description 77
- 238000007781 pre-processing Methods 0.000 claims abstract description 67
- 238000001514 detection method Methods 0.000 claims abstract description 60
- 230000002159 abnormal effect Effects 0.000 claims abstract description 21
- 238000013024 troubleshooting Methods 0.000 claims abstract description 15
- 238000012937 correction Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims description 20
- 238000012360 testing method Methods 0.000 claims description 20
- 230000011664 signaling Effects 0.000 claims description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 9
- 238000009826 distribution Methods 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 7
- 238000004140 cleaning Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000002203 pretreatment Methods 0.000 claims description 3
- 238000011835 investigation Methods 0.000 claims 2
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 238000012423 maintenance Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000003032 molecular docking Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 230000002547 anomalous effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/04—Arrangements for maintaining operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0677—Localisation of faults
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a core network hidden trouble shooting method, a device, electronic equipment and a storage medium, comprising the following steps: accessing a real-time alarm stream, and preprocessing the real-time alarm stream to obtain alarm characteristic data; acquiring data to be judged, and inputting the data to be judged and the alarm characteristic data into an abnormality detection model to obtain an initial abnormality judgment result; and correcting the initial abnormality judgment result based on the nuclear density estimation, and outputting an abnormality positioning result. According to the invention, the alarm flow is preprocessed, network alarm vectorization is performed, an abnormal detection model is input for detection, and finally, correction is performed through nuclear density estimation, so that hidden faults in the network can be found in time, and the judging speed and accuracy are improved.
Description
Technical Field
The present invention relates to the field of network maintenance technologies, and in particular, to a method and apparatus for checking hidden danger of a core network, an electronic device, and a storage medium.
Background
The communication network is divided into a core network, a wireless network, a transmission network and a data network, wherein the core network is the brain of the whole network and is responsible for controlling and distributing the whole service. With the development of NFV, 5G and other technologies, the core network architecture is increasingly complicated, and the core network faults are gradually changed from the traditional dominant to the dominant and recessive combined distribution. Compared with dominant faults, the hidden faults can be directly known by observing alarms, are hidden, and can be found by a series of analysis and diagnosis means similar to the diseases still in the latent period.
Currently, a core network has a large amount of operation data in the current network, and the operation state of the network is reflected by the data from different dimensions. So for a hidden trouble of the core network, it is necessary to start with these network operation data and find the abnormal point therein. The method for discovering the hidden trouble of the core network is mainly divided into two types, namely a service dial testing method and a performance index method:
The service dial testing method is a mainstream method at present, the method simulates the network service used by a user, the dial testing can be successfully completed under the normal condition of the network, and when a fault occurs, the service dial testing can fail and prompt maintenance personnel so as to find the fault; the performance index method is to set a threshold value for KPI indexes of various services in network operation, and when the indexes are lower than the threshold value, relevant alarms are generated and maintenance personnel are prompted so as to find faults.
It can be found that the service dial testing method has the following disadvantages:
1) Network faults discovered by simulating user behaviors cannot be strictly called as hidden trouble fault discovery, and because the faults truly occur in a network and cause partial network service failure, a certain hysteresis exists in a service dial testing method;
2) A large amount of dial test card resources are required to be occupied, real services are required to be frequently executed, and pressure is caused on a network with smaller partial capacity;
3) The actual service charge can be generated, and the cost is relatively high;
4) The introduced network operation data are too little to carry out overview analysis on the overall operation condition of the network.
The performance index method has the following disadvantages:
1) The fault is found by setting a threshold value, but the adaptation capability of the method is poor due to the fixed threshold value, for example, fluctuation of performance indexes is increased due to the decrease of traffic at night, the probability of exceeding the threshold value is improved, and the fixed threshold value cannot consider the factor;
2) Because the network is dynamically developed, a fixed performance threshold cannot always represent a changed network, and therefore the threshold needs to be manually analyzed and corrected frequently;
3) Although network failures can be found, the cause of the failure cannot be further checked.
Disclosure of Invention
The invention provides a method, a device, electronic equipment and a storage medium for checking hidden danger of a core network, which are used for solving the defect that only obvious network faults can be checked in the prior art.
In a first aspect, the present invention provides a method for checking hidden danger of a core network, including:
accessing a real-time alarm stream, and preprocessing the real-time alarm stream to obtain alarm characteristic data;
Acquiring data to be judged, and inputting the data to be judged and the alarm characteristic data into an abnormality detection model to obtain an initial abnormality judgment result;
And correcting the initial abnormality judgment result based on the nuclear density estimation, and outputting an abnormality positioning result.
In one embodiment, the acquiring the real-time alarm stream, preprocessing the real-time alarm stream to obtain alarm feature data specifically includes:
accessing a kafka real-time alarm stream interface, acquiring a total alarm field of a preset system core network, and respectively carrying out denoising, standardization and time slicing treatment on the total alarm field to obtain pre-treatment alarm data;
Vectorizing the pre-processing alarm data, extracting the vectorized pre-processing alarm data according to a preset feature field, and obtaining the alarm feature data with the same dimensionality as the preset feature field;
and classifying the alarm vectorization data according to the performance data, the dial testing data, the engineering data and the resource data to obtain the alarm characteristic data.
In one embodiment, the access kafka real-time alarm stream interface obtains a total alarm field of a preset system core network, and performs denoising, standardization and time slicing processing on the total alarm field to obtain preprocessed alarm data, which specifically includes:
Removing noise alarms corresponding to empty fields or abnormal fields in the full alarm fields to obtain alarm texts;
unifying different expression forms in the same field in the alarm text, and outputting a standardized alarm text;
And slicing and sampling the standardized alarm text according to preset slicing time length to obtain the preprocessing alarm data.
In one embodiment, the obtaining the data to be determined, inputting the data to be determined and the alarm feature data to an anomaly detection model to obtain an initial anomaly determination result specifically includes:
Acquiring data to be judged with preset dimensions, and classifying in combination with the alarm characteristic data to obtain model input data;
and inputting the model input data into the abnormality detection model, and obtaining the initial abnormality judgment result through negative detection, abnormality detection and voting.
In one embodiment, the obtaining the data to be determined with the preset dimension, and classifying in combination with the alert feature data, to obtain model input data specifically includes:
Respectively acquiring performance data, dial testing data, engineering data and resource data to form the data to be judged;
Performing data cleaning on the alarm feature data to generate a new input field set, wherein the input field set comprises size, diff1day, diff7day and pool_diff;
filling the missing data in the data to be judged with 0 to obtain the filled data to be judged;
And classifying the input field set and the complemented data to be determined according to a signaling plane and a user plane to obtain the model input data.
In one embodiment, the correcting the initial anomaly determination result based on the kernel density estimation, and outputting an anomaly localization result specifically includes:
preprocessing the multidimensional data corresponding to the initial abnormality judgment result to obtain preprocessed result data;
Inputting the preprocessing result data into a kernel density KDE model for density fitting to obtain a kernel density probability estimated value;
And correcting the estimated value of the nuclear density probability to obtain the maximum value CHANGE DEGREE in the data dimension, namely the abnormal positioning result.
In one embodiment, the preprocessing the multidimensional data corresponding to the initial anomaly determination result to obtain preprocessed result data specifically includes:
for the signaling surface performance index, obtaining signaling surface pretreatment result data based on subtracting an index original value from 1;
For the user plane performance index, obtaining user plane preprocessing result data based on an index original value, an index maximum value and an index minimum value;
And for the alarm type index, based on the original index value, the maximum index value and the minimum index value, carrying out Gaussian distribution conversion to obtain alarm type preprocessing result data.
In a second aspect, the present invention further provides a core network hidden trouble shooting device, including:
the preprocessing module is used for accessing the real-time alarm stream, preprocessing the real-time alarm stream and obtaining alarm characteristic data;
the processing module is used for acquiring data to be judged, inputting the data to be judged and the alarm characteristic data into an abnormality detection model, and obtaining an initial abnormality judgment result;
And the correction module is used for correcting the initial abnormality judgment result based on the kernel density estimation and outputting an abnormality positioning result.
In a third aspect, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the steps of any one of the core network hidden trouble shooting methods described above when executing the program.
In a fourth aspect, the present invention also provides a non-transitory computer readable storage medium, on which is stored a computer program which, when executed by a processor, implements the steps of any one of the core network hidden trouble shooting methods described above.
According to the core network hidden trouble shooting method, the device, the electronic equipment and the storage medium, through preprocessing the alarm stream, carrying out network alarm vectorization, inputting the abnormal detection model for detection, and finally correcting through the kernel density estimation, hidden faults in the network can be found in time, and judging speed and accuracy are improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a method for checking hidden danger of a core network provided by the invention;
FIG. 2 is a schematic diagram of an alarm data processing procedure provided by the present invention;
FIG. 3 is a flow chart of the anomaly detection model computation provided by the present invention;
FIG. 4 is a flow chart of a nuclear density estimation provided by the present invention;
FIG. 5 is a flow chart of the correction and output of the nuclear density probability provided by the present invention;
Fig. 6 is a schematic structural diagram of a core network hidden trouble shooting device provided by the present invention;
Fig. 7 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a flow chart of a method for checking hidden danger of a core network, which is provided by the invention, as shown in fig. 1, and includes:
101, accessing a real-time alarm stream, and preprocessing the real-time alarm stream to obtain alarm characteristic data;
102, acquiring data to be judged, and inputting the data to be judged and the alarm characteristic data into an abnormality detection model to obtain an initial abnormality judgment result;
And 103, correcting the initial abnormality judgment result based on the nuclear density estimation, and outputting an abnormality positioning result.
Specifically, the invention provides a method for carrying out pretreatment on the real-time alarm stream based on the network alarm stream, and provides an alarm vectorization method, which converts the alarm from a text file to a vector model which can be applied to mathematical calculation, in order to discover the necessity of network hidden trouble as soon as possible, designs an abnormality detection model, and simultaneously provides a processing method of model input data, in addition, through setting voting logic, the method is expected to reduce instability caused by an anomaly detection algorithm, and is also expected to provide a method for finding the root cause of the hidden trouble through nuclear density estimation for finding the main hidden trouble point in the hidden trouble of the fault, a processing method of input data of a nuclear density estimation model is designed, and finally, a data post-correction method of expert experience is combined, so that negative influence of certain specific scenes on accuracy in data analysis can be shielded through the method.
It can be understood that the more timely the hidden trouble fault is found means that the more the network is blocked, the smaller the business influence of the user is, the time of network faults can be reduced by shortening the time of hidden trouble fault finding, and the aim of reducing cost and enhancing efficiency is fulfilled. At present, according to the condition of the average fault quantity and service loss in a certain place and month, the hidden danger discovery time length is shortened by 10 minutes, and the cost can be saved by about one hundred and forty thousand yuan in each year.
According to the invention, the alarm flow is preprocessed, network alarm vectorization is performed, an abnormal detection model is input for detection, and finally, correction is performed through nuclear density estimation, so that hidden faults in the network can be found in time, and the judging speed and accuracy are improved.
Based on the above embodiment, step 101 in the method specifically includes:
accessing a kafka real-time alarm stream interface, acquiring a total alarm field of a preset system core network, and respectively carrying out denoising, standardization and time slicing treatment on the total alarm field to obtain pre-treatment alarm data;
Vectorizing the pre-processing alarm data, extracting the vectorized pre-processing alarm data according to a preset feature field, and obtaining the alarm feature data with the same dimensionality as the preset feature field;
and classifying the alarm vectorization data according to the performance data, the dial testing data, the engineering data and the resource data to obtain the alarm characteristic data.
The access kafka real-time alarm stream interface acquires a total alarm field of a preset system core network, and respectively performs denoising, standardization and time slicing treatment on the total alarm field to obtain preprocessing alarm data, which specifically comprises the following steps:
Removing noise alarms corresponding to empty fields or abnormal fields in the full alarm fields to obtain alarm texts;
unifying different expression forms in the same field in the alarm text, and outputting a standardized alarm text;
And slicing and sampling the standardized alarm text according to preset slicing time length to obtain the preprocessing alarm data.
Specifically, firstly, the real-time alarm stream is accessed, standardized and preprocessed:
Interfacing with a kafka real-time alarm stream interface of a fault management system, and accessing an alarm and related total alarm field of a 5GC, NFV, EPC three-domain core network; the noise alarm with the field value being null or the field value being abnormal is removed, and the method has the significance that the number of alarms to be calculated can be reduced, and the operation pressure is reduced; carrying out alarm standardization operation, carrying out unified standardization on different expression forms possibly existing in the same field in an alarm text, and uniformly modifying the expression forms into Chinese name forms if the machine room information comprises two expression forms of machine room codes and machine room Chinese names; the access alarm is subjected to 5-minute slicing preprocessing operation, sampling work is carried out every 5 minutes according to the alarm occurrence time, and if the alarm occurrence time is 0:00-0: the alarms of 05 would constitute one set, 12 x 24 = 288 sets throughout the day, as shown in fig. 2.
Then, the alarm vectorization is performed, because the alarm data is continuous text, in order to facilitate the operation of the system, the alarm needs to be reconstructed into a vector model which can be calculated. Meanwhile, because the alarm contains a plurality of fields, wherein part of fields (such as manufacturer alarm ID and work list number) have no value of data analysis and calculation, the characteristic field which can represent the alarm characteristic needs to be selected from all the fields and constructed into an N-dimensional vector, so that quick and efficient calculation is realized.
The following feature fields are extracted:
The method comprises the steps of alarm occurrence time, primary specialty, secondary specialty, equipment type, province, local market, manufacturer, alarm title, network element name, alarm positioning object name, circuit name, alarm unique identification, alarm clearing state, network element alias, alarm eliminating time, monitoring level, alarm level, opposite network element and service system.
The above 19 feature fields, each representing an attribute of the alarm. Constructing all fields as a 19-dimensional vector creates an absolutely unique alert for subsequent computation.
And then accessing the performance data, the dial testing data, the engineering data and the resource data respectively:
And the real-time performance data interface of the docking performance management platform receives the performance file in an ftp mode, wherein the performance file comprises performance data of 5 minutes of granularity of 4/5G signaling plane and user plane network elements. The performance data are shown in table 1:
TABLE 1
The dial testing alarm data interface of the docking fault management system consumes data in a kafka mode and comprises 4G EPC domain dial testing alarm data;
Receiving a file by an ftp mode through an engineering cut-over data interface of a cut-over network operation and maintenance management platform, wherein the file comprises engineering information in the EPC, vEPC and 5GC network fields;
And receiving a file by an ftp mode, wherein the file comprises EPC MME/SAEGW pool information and NFV/5GC longitudinal association resource information by a network topology resource interface of the docking resource management system.
Based on any of the above embodiments, step 102 of the method specifically includes:
Acquiring data to be judged with preset dimensions, and classifying in combination with the alarm characteristic data to obtain model input data;
and inputting the model input data into the abnormality detection model, and obtaining the initial abnormality judgment result through negative detection, abnormality detection and voting.
The method for obtaining the data to be determined with the preset dimension includes the steps of classifying in combination with the alarm feature data to obtain model input data, and specifically includes the following steps:
Respectively acquiring performance data, dial testing data, engineering data and resource data to form the data to be judged;
Performing data cleaning on the alarm feature data to generate a new input field set, wherein the input field set comprises size, diff1day, diff7day and pool_diff;
filling the missing data in the data to be judged with 0 to obtain the filled data to be judged;
And classifying the input field set and the complemented data to be determined according to a signaling plane and a user plane to obtain the model input data.
Specifically, for the foregoing input alarm data, a new input field is formed after data cleaning and processing, and then the input field enters the anomaly detection model:
1) Size: the field represents the alarm quantity in the current 5-minute time slice, the processing mode of the field is to group the alarm sets obtained through the slice preprocessing operation according to the characteristic field 'network element names', alarms with the same 'network element names' form a set, meanwhile, alarms with the alarm grades not being 1,2 and 3 are deleted, and then whether each alarm in the time slice has clearing time is judged, namely, if the alarm is cleared in the slice time, the corresponding alarm is deleted. Finally, calculating the total alarm quantity in the set and assigning the total alarm quantity to the size;
2) diff1day: this field represents the difference between the alarm amount of the current 5-minute time slice and the alarm amount of the 5-minute time slice in the same time period of yesterday, and the calculation formula is as follows:
diff1day＝sizetoday-sizeyesterday (1)
wherein size today is the current size value and size yesterday is the yesterday same time size value;
3) diff7day: this field represents the difference between the alarm amount of the current 5-minute time slice and the alarm amount of the 5-minute time slice in the previous period, and the calculation formula is as follows:
diff7day＝sizetoday-sizelastweek (2)
size yesterday is the last week time size value;
4) pool_diff: this field represents the difference between the size of the home network element size value and the average size value in pool, and the calculation formula is:
pool_diff＝sizetoday-sizepoolava (4)
wherein size poolava is average size in pool, n is the number of pool network elements, and Σ pool size is the sum of the sizes of each network element in pool.
In addition, for the performance data input in the foregoing embodiment, there are cases where the data file is lost, and the missing data is complemented by a 0-value padding method.
Classifying the processed data according to a signaling plane and a user plane to respectively obtain 9-dimensional input data of the signaling plane and 8-dimensional input data of the user plane, as shown in table 2:
TABLE 2
Inputting the data into an anomaly detection model for calculation, wherein the data passes through the following modules:
1) And (5) a negative removing module: since diff1day and diff7day represent the difference between the current alarm amount and the history, and a negative value means that the alarm amount is reduced, and the network condition is improved, so that the network condition should not be set as abnormal at this time, and when the data passes through the negative module, the negative values of diff1day, diff7day and pool_diff are set as 0;
2) An abnormality detection module: using PYOD tool library, the selected algorithm contained the following 12 specific anomaly detection algorithms, as shown in table 3:
TABLE 3 Table 3
Outputting a 12-dimensional variable after model calculation, wherein each dimension is an abnormal result judged by an abnormal detection algorithm;
3) And a voting module: the present module votes on the result output by the anomaly detection module, and when two or more anomaly detection algorithms consider that the piece of data is anomalous, the voting module finally determines that the piece of data is anomalous, as shown in fig. 3.
Based on any of the above embodiments, step 103 of the method specifically includes:
preprocessing the multidimensional data corresponding to the initial abnormality judgment result to obtain preprocessed result data;
Inputting the preprocessing result data into a kernel density KDE model for density fitting to obtain a kernel density probability estimated value;
And correcting the estimated value of the nuclear density probability to obtain the maximum value CHANGE DEGREE in the data dimension, namely the abnormal positioning result.
The preprocessing of the multidimensional data corresponding to the initial abnormality judgment result to obtain preprocessed result data specifically includes:
for the signaling surface performance index, obtaining signaling surface pretreatment result data based on subtracting an index original value from 1;
For the user plane performance index, obtaining user plane preprocessing result data based on an index original value, an index maximum value and an index minimum value;
And for the alarm type index, based on the original index value, the maximum index value and the minimum index value, carrying out Gaussian distribution conversion to obtain alarm type preprocessing result data.
Specifically, the invention detects the abnormality of the input data and outputs the abnormality judgment, but because of the voting mechanism, it is not determined which data dimension is the main cause of the abnormality of the equipment, so that the main abnormality dimension judgment is needed, and the multidimensional data is preprocessed for subsequent dimension abnormality analysis.
1) For the signaling surface performance indexes of TAU success rate ', ' attachment success rate ', ' service request success rate ', ' X2 switching success rate ', ' paging success rate ', the user surface performance index ' HTTP session response success rate is more than 500KB ', preprocessing calculation is carried out by adopting a formula (5):
p′＝1-p (5)
wherein p is an original value, and p' is an input value of a subsequent model after pretreatment;
2) For the user plane performance index 'HTTP session average response time delay is larger than 500KB (ms)', HTTP download rate is larger than 500KB (Kbps) 'and HTTP500 downlink traffic (MB)', adopting a formula (6) to perform normalization processing:
Wherein x is an original value, x min is a minimum value of the dimension within 7 days, x max is a maximum value within 7 days, and x' is an input value of a subsequent model after pretreatment;
3) For the alarm class data 'size', 'diff1day', 'diff7day', 'pool_diff', normalization processing is performed first:
Wherein a is the original value, a min is the minimum value of the dimension within 7 days, a max is the maximum value within 7 days, and a' is the normalized value;
Considering that the business meaning of size, namely the network element alarm quantity, meets poisson distribution, the evolution process converts the indexes into Gaussian distribution and reserves symbols:
where a' is the normalized value from equation 7 and a "is the value after gaussian transformation.
And (3) inputting the preprocessed data into a kernel density KDE model for density fitting, wherein the time length of the input data is approximately 7 days, and the total number of the input data is 7×12×24=2016. Since the data has been transformed into a gaussian distribution, the kernel density function of the selected gaussian kernel continues to calculate:
Where K (x) is a kernel density function, here a Gaussian kernel density function is used. x is input data, h is bandwidth (bandwidth), which is another parameter of model training, where the determination is made using equation (10):
Wherein R (k), m 2 (k) satisfy:
R(k)＝∫K(x)2dx (11)
m2(k)＝∫x2K(x)2dx (12)
n is the number of input data, 2016.
The kernel density estimation can be performed by using the formula (9) so far, and CHANGE DEGREE of the input data, i.e. the probability in the kernel density, is calculatedThe flow is shown in fig. 4.
Since the nuclear density is estimated as pure data analysis, it is also corrected in combination with curing experience during the outcome output to increase the outcome accuracy.
1) For signaling plane performance indexes of 'TAU success rate', 'attach success rate', 'service request success rate', 'X2 handover success rate', 'paging success rate', user plane performance index 'HTTP session response success rate is greater than 500KB', 'HTTP download rate is greater than 500KB (Kbps)', and CHANGE DEGREE is set to 0 when the 7 indexes have original values greater than 7-day average;
2) For the user plane performance index 'HTTP session average response time delay is more than 500KB (ms)', when the original value of the index is less than the historical 7-day average value, setting the corresponding CHANGE DEGREE to 0;
3) For the user plane performance index 'HTTP500 downstream traffic (MB)', let f now be the today traffic and f yesterday be the yesterday time traffic, when the following is satisfied At this time CHANGE DEGREE is set to 0.
After correction, CHANGE DEGREE of each data dimension is compared, wherein the dimension with the largest value is the main factor causing the abnormality, namely the main hidden trouble point, and the correction flow is shown in fig. 5.
The core network hidden trouble shooting device provided by the invention is described below, and the core network hidden trouble shooting device described below and the core network hidden trouble shooting method described above can be correspondingly referred to each other.
Fig. 6 is a schematic structural diagram of a core network hidden trouble shooting device provided by the present invention, as shown in fig. 6, including: a preprocessing module 61, a processing module 62, and a correction module 63; wherein:
The preprocessing module 61 is used for accessing a real-time alarm stream, and preprocessing the real-time alarm stream to obtain alarm characteristic data; the processing module 62 is configured to obtain data to be determined, input the data to be determined and the alarm feature data to an anomaly detection model, and obtain an initial anomaly determination result; the correction module 63 is configured to correct the initial anomaly determination result based on the kernel density estimation, and output an anomaly localization result.
According to the invention, the alarm flow is preprocessed, network alarm vectorization is performed, an abnormal detection model is input for detection, and finally, correction is performed through nuclear density estimation, so that hidden faults in the network can be found in time, and the judging speed and accuracy are improved.
Based on any of the above embodiments, the preprocessing module 61 includes: a first preprocessing sub-module 611, a vectoring sub-module 612, and a classification sub-module 613, wherein:
The first preprocessing sub-module 611 is configured to access the kafka real-time alarm stream interface, obtain a full alarm field of a preset system core network, and perform denoising, normalization and time slicing processing on the full alarm field to obtain preprocessed alarm data; the vectorization sub-module 612 is configured to vectorize the pre-processed alarm data, extract the vectorized pre-processed alarm data according to a preset feature field, and obtain the alarm feature data having the same dimension as the preset feature field; the classifying sub-module 613 is configured to classify the alarm vectorization data according to performance data, dial testing data, engineering data and resource data, so as to obtain the alarm feature data.
Based on any of the above embodiments, the first preprocessing sub-module 611 is specifically configured to:
Removing noise alarms corresponding to empty fields or abnormal fields in the full alarm fields to obtain alarm texts; unifying different expression forms in the same field in the alarm text, and outputting a standardized alarm text; and slicing and sampling the standardized alarm text according to preset slicing time length to obtain the preprocessing alarm data.
Based on any of the above embodiments, the processing module 62 includes: a second preprocessing sub-module 621 and a processing sub-module 622, wherein:
The second preprocessing sub-module 621 is configured to obtain data to be determined having a preset dimension, and classify the data in combination with the alarm feature data to obtain model input data; the processing sub-module 622 is configured to input the model input data into the anomaly detection model, and obtain the initial anomaly determination result through negative anomaly detection and voting.
Based on any of the above embodiments, the second preprocessing sub-module 621 is specifically configured to:
Respectively acquiring performance data, dial testing data, engineering data and resource data to form the data to be judged; performing data cleaning on the alarm feature data to generate a new input field set, wherein the input field set comprises size, diff1day, diff7day and pool_diff; filling the missing data in the data to be judged with 0 to obtain the filled data to be judged; and classifying the input field set and the complemented data to be determined according to a signaling plane and a user plane to obtain the model input data.
Based on any of the above embodiments, the correction module 63 includes: a third preprocessing sub-module 631, an estimation sub-module 632, and a correction sub-module 633, wherein:
The third preprocessing sub-module 631 is configured to preprocess the multidimensional data corresponding to the initial anomaly determination result to obtain preprocessed result data; the estimation sub-module 632 is used for inputting the preprocessing result data into a kernel density KDE model for density fitting to obtain a kernel density probability estimation value; the correction sub-module 633 is configured to correct the estimated value of the kernel density probability to obtain a maximum value CHANGE DEGREE in a data dimension, that is, the abnormal positioning result.
Based on any of the above embodiments, the third preprocessing sub-module 631 is specifically configured to:
For the signaling surface performance index, obtaining signaling surface pretreatment result data based on subtracting an index original value from 1; for the user plane performance index, obtaining user plane preprocessing result data based on an index original value, an index maximum value and an index minimum value; and for the alarm type index, based on the original index value, the maximum index value and the minimum index value, carrying out Gaussian distribution conversion to obtain alarm type preprocessing result data.
Fig. 7 illustrates a physical schematic diagram of an electronic device, as shown in fig. 7, which may include: processor 710, communication interface (CommunicationsInterface) 720, memory 730, and communication bus 740, wherein processor 710, communication interface 720, memory 730 communicate with each other via communication bus 740. Processor 710 may invoke logic instructions in memory 730 to perform a core network hidden trouble shooting method comprising: accessing a real-time alarm stream, and preprocessing the real-time alarm stream to obtain alarm characteristic data; acquiring data to be judged, and inputting the data to be judged and the alarm characteristic data into an abnormality detection model to obtain an initial abnormality judgment result; and correcting the initial abnormality judgment result based on the nuclear density estimation, and outputting an abnormality positioning result.
Further, the logic instructions in the memory 730 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, randomAccessMemory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer readable storage medium, the computer program including program instructions which, when executed by a computer, enable the computer to perform the core network hidden trouble shooting method provided by the above methods, the method comprising: accessing a real-time alarm stream, and preprocessing the real-time alarm stream to obtain alarm characteristic data; acquiring data to be judged, and inputting the data to be judged and the alarm characteristic data into an abnormality detection model to obtain an initial abnormality judgment result; and correcting the initial abnormality judgment result based on the nuclear density estimation, and outputting an abnormality positioning result.
In still another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, is implemented to perform the above-provided core network hidden trouble shooting methods, the method comprising: accessing a real-time alarm stream, and preprocessing the real-time alarm stream to obtain alarm characteristic data; acquiring data to be judged, and inputting the data to be judged and the alarm characteristic data into an abnormality detection model to obtain an initial abnormality judgment result; and correcting the initial abnormality judgment result based on the nuclear density estimation, and outputting an abnormality positioning result.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (7)
1. The hidden danger investigation method for the core network is characterized by comprising the following steps:
accessing a real-time alarm stream, and preprocessing the real-time alarm stream to obtain alarm characteristic data;
Acquiring data to be judged, and inputting the data to be judged and the alarm characteristic data into an abnormality detection model to obtain an initial abnormality judgment result;
Correcting the initial abnormality judgment result based on the kernel density estimation, and outputting an abnormality positioning result;
the accessing the real-time alarm stream, preprocessing the real-time alarm stream to obtain alarm feature data, specifically includes:
Accessing a kafka real-time alarm stream interface, acquiring a total alarm field of a preset system core network, and respectively carrying out denoising, standardization and time slicing treatment on the total alarm field to obtain pre-treatment alarm data; the time slicing process is text slicing sampling process carried out according to preset slicing duration;
vectorizing the pre-processing alarm data, extracting vectorized pre-processing alarm data according to a preset characteristic field, and obtaining alarm vectorized data with the same dimensionality as the preset characteristic field;
classifying the alarm vectorization data according to performance data, dial testing data, engineering data and resource data to obtain alarm characteristic data;
The obtaining the data to be determined, inputting the data to be determined and the alarm feature data into an anomaly detection model to obtain an initial anomaly determination result, specifically including:
Acquiring data to be judged with preset dimensions, and classifying in combination with the alarm characteristic data to obtain model input data;
Inputting the model input data into the abnormality detection model, and obtaining the initial abnormality judgment result through negative abnormality detection and voting;
the method for obtaining the data to be determined with the preset dimension includes the steps of classifying in combination with the alarm feature data to obtain model input data, and specifically includes the following steps:
Respectively acquiring performance data, dial testing data, engineering data and resource data to form the data to be judged;
Performing data cleaning on the alarm feature data to generate a new input field set, wherein the input field set comprises size, diff1day, diff7day and pool_diff; the Size field represents the alarm amount in the current 5-minute time slice, the diff1day field represents the difference value between the alarm amount of the current 5-minute time slice and the alarm amount of the 5-minute time slice in the same time period of yesterday, the diff7day field represents the difference value between the alarm amount of the current 5-minute time slice and the alarm amount of the 5-minute time slice in the previous time period, and the pool_diff field represents the difference value between the Size value of the local network element and the average Size value in the pool;
filling the missing data in the data to be judged with 0 to obtain the filled data to be judged;
Classifying the input field set and the complemented data to be judged according to a signaling plane and a user plane to obtain the model input data;
the model input data is input to the abnormality detection model, and the initial abnormality judgment result is obtained through negative detection, abnormality detection and voting, specifically comprising:
Inputting the model input data into an anomaly detection model for calculation, and processing the model input data through the following modules: a negative removing module, configured to set negative values of diff1day, diff7day, pool_diff to 0; the abnormality detection module is used for carrying out abnormality detection on the result output by the negative removal module by adopting a plurality of abnormality detection algorithms selected from PYOD tool libraries to obtain a multi-dimensional variable, wherein each dimension of the multi-dimensional variable is an abnormality result judged by an abnormality detection algorithm; and the voting module is used for voting the result output by the abnormality detection module, and judging the current model input data as abnormal when more than two abnormality detection algorithms judge that the current model input data is abnormal.
2. The method for checking hidden danger of a core network according to claim 1, wherein the accessing the kafka real-time alarm stream interface obtains a total alarm field of a preset system core network, and performs denoising, standardization and time slicing processing on the total alarm field to obtain pre-processing alarm data, specifically comprising:
Removing noise alarms corresponding to empty fields or abnormal fields in the full alarm fields to obtain alarm texts;
unifying different expression forms in the same field in the alarm text, and outputting a standardized alarm text;
And slicing and sampling the standardized alarm text according to preset slicing time length to obtain the preprocessing alarm data.
3. The method for inspecting hidden danger of a core network according to claim 1, wherein the correcting the initial anomaly determination result based on the core density estimation, and outputting an anomaly locating result, specifically comprises:
preprocessing the multidimensional data corresponding to the initial abnormality judgment result to obtain preprocessed result data;
Inputting the preprocessing result data into a kernel density KDE model for density fitting to obtain a kernel density probability estimated value;
And correcting the estimated value of the nuclear density probability to obtain the maximum value CHANGE DEGREE in the data dimension, namely the abnormal positioning result.
4. The method for inspecting hidden danger of a core network according to claim 3, wherein the preprocessing the multidimensional data corresponding to the initial anomaly determination result to obtain preprocessed result data specifically comprises:
for the signaling surface performance index, obtaining signaling surface pretreatment result data based on subtracting an index original value from 1;
For the user plane performance index, obtaining user plane preprocessing result data based on an index original value, an index maximum value and an index minimum value;
And for the alarm type index, based on the original index value, the maximum index value and the minimum index value, carrying out Gaussian distribution conversion to obtain alarm type preprocessing result data.
5. Core network hidden danger investigation device, its characterized in that includes:
the preprocessing module is used for accessing the real-time alarm stream, preprocessing the real-time alarm stream and obtaining alarm characteristic data;
the processing module is used for acquiring data to be judged, inputting the data to be judged and the alarm characteristic data into an abnormality detection model, and obtaining an initial abnormality judgment result;
the correction module is used for correcting the initial abnormality judgment result based on the kernel density estimation and outputting an abnormality positioning result;
wherein, the preprocessing module includes:
the first preprocessing submodule is used for accessing the kafka real-time alarm stream interface, acquiring the total alarm field of a preset system core network, and respectively carrying out denoising, standardization and time slicing on the total alarm field to obtain preprocessing alarm data; the time slicing process is text slicing sampling process carried out according to preset slicing duration;
the vectorization sub-module is used for vectorizing the pre-processing alarm data, extracting vectorized pre-processing alarm data according to a preset characteristic field, and obtaining alarm vectorization data with the same dimension as the preset characteristic field;
the classifying sub-module is used for classifying the alarm vectorization data according to performance data, dial testing data, engineering data and resource data to obtain the alarm characteristic data;
wherein the processing module comprises:
The second preprocessing sub-module is used for acquiring data to be judged with preset dimensions, and classifying the data in combination with the alarm characteristic data to obtain model input data;
the processing sub-module is used for inputting the model input data into the abnormality detection model, and obtaining the initial abnormality judgment result through negative detection, abnormality detection and voting;
The second preprocessing sub-module is specifically configured to: respectively acquiring performance data, dial testing data, engineering data and resource data to form the data to be judged; performing data cleaning on the alarm feature data to generate a new input field set, wherein the input field set comprises size, diff1day, diff7day and pool_diff; the Size field represents the alarm amount in the current 5-minute time slice, the diff1day field represents the difference value between the alarm amount of the current 5-minute time slice and the alarm amount of the 5-minute time slice in the same time period of yesterday, the diff7day field represents the difference value between the alarm amount of the current 5-minute time slice and the alarm amount of the 5-minute time slice in the previous time period, and the pool_diff field represents the difference value between the Size value of the local network element and the average Size value in the pool; filling the missing data in the data to be judged with 0 to obtain the filled data to be judged; classifying the input field set and the complemented data to be judged according to a signaling plane and a user plane to obtain the model input data;
The processing sub-module is specifically configured to: inputting the model input data into an anomaly detection model for calculation, and processing the model input data through the following modules: a negative removing module, configured to set negative values of diff1day, diff7day, pool_diff to 0; the abnormality detection module is used for carrying out abnormality detection on the result output by the negative removal module by adopting a plurality of abnormality detection algorithms selected from PYOD tool libraries to obtain a multi-dimensional variable, wherein each dimension of the multi-dimensional variable is an abnormality result judged by an abnormality detection algorithm; and the voting module is used for voting the result output by the abnormality detection module, and judging the current model input data as abnormal when more than two abnormality detection algorithms judge that the current model input data is abnormal.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the core network hidden trouble shooting method according to any one of claims 1 to 4 when executing the computer program.
7. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the core network hidden trouble shooting method according to any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110387834.2A CN115209452B (en) | 2021-04-09 | 2021-04-09 | Method and device for checking hidden danger of core network, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110387834.2A CN115209452B (en) | 2021-04-09 | 2021-04-09 | Method and device for checking hidden danger of core network, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115209452A CN115209452A (en) | 2022-10-18 |
| CN115209452B true CN115209452B (en) | 2024-11-08 |
Family
ID=83571213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110387834.2A Active CN115209452B (en) | 2021-04-09 | 2021-04-09 | Method and device for checking hidden danger of core network, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115209452B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116827817B (en) * | 2023-04-12 | 2024-09-24 | 国网河北省电力有限公司信息通信分公司 | Data link state monitoring method, device, monitoring system and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101035159A (en) * | 2007-02-12 | 2007-09-12 | 中国移动通信集团浙江有限公司 | Dialed monitoring system for the troubleshooting of the communication network and its monitoring method |
| CN103812688A (en) * | 2012-11-15 | 2014-05-21 | 中国移动通信集团设计院有限公司 | Alarm determining method and device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104066103B (en) * | 2013-03-19 | 2017-10-10 | 中国移动通信集团湖北有限公司 | The Fault Locating Method and device of data service core net |
| EP3021520B1 (en) * | 2013-09-18 | 2017-11-15 | Huawei Technologies Co., Ltd. | Method and apparatus for delimiting problem |
| US9544812B2 (en) * | 2015-01-30 | 2017-01-10 | Alcatel Lucent | System and method for mitigating network congestion using fast congestion detection in a wireless radio access network (RAN) |
| EP3131234B1 (en) * | 2015-08-14 | 2018-05-02 | Accenture Global Services Limited | Core network analytics system |
| CN108605035B (en) * | 2016-07-18 | 2021-07-30 | 荣耀终端有限公司 | Processing method and terminal for abnormal hanging up of core network |
| CN108156037B (en) * | 2017-12-29 | 2020-12-15 | 中国移动通信集团江苏有限公司 | Alarm correlation analysis method, device, equipment and medium |
| US10484892B2 (en) * | 2018-02-20 | 2019-11-19 | Verizon Patent And Licensing Inc. | Contextualized network optimization |
-
2021
- 2021-04-09 CN CN202110387834.2A patent/CN115209452B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101035159A (en) * | 2007-02-12 | 2007-09-12 | 中国移动通信集团浙江有限公司 | Dialed monitoring system for the troubleshooting of the communication network and its monitoring method |
| CN103812688A (en) * | 2012-11-15 | 2014-05-21 | 中国移动通信集团设计院有限公司 | Alarm determining method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115209452A (en) | 2022-10-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112685324B (en) | Method and system for generating test scheme | |
| CN111309565B (en) | Alarm processing method and device, electronic equipment and computer readable storage medium | |
| CN110471913A (en) | A kind of data cleaning method and device | |
| CN110457175B (en) | Service data processing method and device, electronic equipment and medium | |
| CN109063433B (en) | False user identification method and device and readable storage medium | |
| CN116184988A (en) | Multi-mode data-based fault prediction method, device, equipment and storage medium | |
| CN112988509A (en) | Alarm message filtering method and device, electronic equipment and storage medium | |
| CN112379325A (en) | Fault diagnosis method and system for intelligent electric meter | |
| CN115209452B (en) | Method and device for checking hidden danger of core network, electronic equipment and storage medium | |
| CN115269288A (en) | Fault determination method, device, equipment and storage medium | |
| CN112416800A (en) | Intelligent contract testing method, device, equipment and storage medium | |
| CN114765574A (en) | Network anomaly delimitation positioning method and device | |
| CN111723182B (en) | Key information extraction method and device for vulnerability text | |
| CN113112126A (en) | Water supply pipe leakage early warning method and system based on artificial intelligence | |
| CN114202110B (en) | Service fault prediction method and device based on RF-XGBOOST | |
| CN117389821A (en) | Log abnormality detection method, device and storage medium | |
| CN113569879B (en) | Training method of abnormal recognition model, abnormal account recognition method and related device | |
| CN115116594B (en) | Method and device for detecting effectiveness of medical device | |
| CN117235588A (en) | Decision tree-based aeroengine gas circuit fault diagnosis method and device | |
| CN117272123B (en) | Sensitive data processing method and device based on large model and storage medium | |
| CN117194275B (en) | Automatic software automatic test plan generation method and system based on intelligent algorithm | |
| CN116107859B (en) | Container fault prediction method and device, electronic equipment and storage medium | |
| CN118585802A (en) | Method, device, electronic equipment and storage medium for determining super parameters of training model | |
| CN114969335A (en) | Abnormal log detection method and device, electronic equipment and readable storage medium | |
| CN115509941A (en) | Data interface testing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |