CN116610642B - Log auditing method and system for multiple types of equipment - Google Patents
Log auditing method and system for multiple types of equipment Download PDFInfo
- Publication number
- CN116610642B CN116610642B CN202310385033.1A CN202310385033A CN116610642B CN 116610642 B CN116610642 B CN 116610642B CN 202310385033 A CN202310385033 A CN 202310385033A CN 116610642 B CN116610642 B CN 116610642B
- Authority
- CN
- China
- Prior art keywords
- log
- equipment
- hazard
- important
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000002159 abnormal effect Effects 0.000 claims abstract description 158
- 238000012544 monitoring process Methods 0.000 claims abstract description 55
- 238000004458 analytical method Methods 0.000 claims abstract description 43
- 230000006870 function Effects 0.000 claims description 24
- 230000000977 initiatory effect Effects 0.000 claims description 18
- 230000005856 abnormality Effects 0.000 claims description 11
- 238000012550 audit Methods 0.000 claims description 11
- 238000007619 statistical method Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 238000003491 array Methods 0.000 claims description 3
- BTCSSZJGUNDROE-UHFFFAOYSA-N gamma-aminobutyric acid Chemical compound NCCCC(O)=O BTCSSZJGUNDROE-UHFFFAOYSA-N 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000009471 action Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Mathematics (AREA)
- Mathematical Physics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Operations Research (AREA)
- Algebra (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a log auditing method of multi-type equipment, which comprises the following steps: the method comprises the steps of constructing a multi-device log monitoring list, carrying out log monitoring on corresponding devices through the multi-device log monitoring list, carrying out scanning analysis on log monitoring content, judging important log information, associating a plurality of first labels with the important log information, establishing a log information pool aiming at different types of log types, determining the log information pool to which the important log information belongs based on analysis of the types of the important log information, determining the log information pool to be scanned and analyzed based on the robbed log abnormal characteristics, carrying out comparison analysis on the first labels of the important log information in the log information pool and the multi-device log abnormal characteristics, judging the coincidence condition, and determining the hazard condition of a current system according to the coincidence condition.
Description
Technical Field
The invention relates to the technical field of log audit, in particular to a log audit method and system for multi-type equipment.
Background
With the development of modern technology, many organizations and businesses already have a large number of devices and systems that may run different operating systems, applications, services, and protocols. Due to the complexity of the equipment and systems, the occurrence of faults and safety events is unavoidable. To ensure the security and stability of devices and systems, log auditing becomes an essential part.
In the prior art, a common method is to use a log collector, the collector can collect logs from different devices and systems and send the logs to a unified position for processing, so that the logs of different devices are processed uniformly, but the method is limited to finding log abnormality and cannot judge system problems shown by various log abnormality, so that in order to improve the running safety of the whole system, a method capable of conducting deeper audit on the logs is needed.
Disclosure of Invention
The invention aims to provide a log auditing method for multi-type equipment, which is used for determining the current problem of a system and comprises the following steps:
Constructing a multi-device log monitoring list, and carrying out log monitoring on corresponding devices based on the multi-device log monitoring list;
Scanning and analyzing the monitored content of the log, determining important log information based on a preset important log characteristic judgment rule, and associating a first tag group with the important log information based on the content of the important log information, wherein the first tag group comprises a plurality of first tags;
establishing a log information pool aiming at different types of log information, determining a log information pool to which the important log information belongs based on type analysis of the important log information, and classifying the important log information;
Carrying out statistical analysis on each system hazard condition, and determining multi-equipment log abnormal characteristics of each system hazard condition;
Determining a plurality of log information pools needing scanning analysis based on the multi-equipment log abnormal characteristics determined during system operation, and determining the current system hazard condition based on the coincidence condition of a first label associated with important log information in the log information pools and the multi-equipment log abnormal characteristics;
in some embodiments of the present application, constructing a multi-device log monitoring directory table, and performing log monitoring on a corresponding device based on the multi-device log monitoring directory table, including:
establishing a system application equipment list, and determining the functions to be executed by the application equipment according to the system operation requirements;
determining a log to be monitored based on a function to be executed by the application equipment, and generating an equipment log monitoring list based on the corresponding situation of the log and the application equipment;
in some embodiments of the present application, a scan analysis is performed on a monitored content of a log, and important log information is determined based on a preset important log feature judgment rule, and a first tag group is associated with the important log information based on the content of the important log information, where the first tag group includes a plurality of first tags, and includes:
Gradually scanning the logs, intercepting the logs conforming to the important log characteristic judgment rules, and generating important log information;
extracting dynamic indexes in the important log information content and generating an index class first label;
extracting the occurrence time in the important log information content and generating a time-class first label;
extracting the equipment marks in the important log information content and generating a first equipment type label;
Extracting the function module marks in the important log information content and generating a function class first label;
combining the index type first tag, the time type first tag, the equipment type first tag and the function type first tag to generate a first tag group;
And establishing an association relation between the first tag group and the important log information.
The method for extracting the dynamic index, the occurrence time, the equipment mark and the functional module mark in the important log information content comprises the following steps:
Establishing a log content identification table aiming at the dynamic index, the occurrence time, the equipment mark and the feature identification code of the function module mark in each log;
and carrying out scanning analysis on the important log information based on the log content identification table, and extracting information of dynamic indexes, occurrence time, equipment marks and functional module marks.
In some embodiments of the present application, performing a statistical analysis on each system hazard situation, determining multi-device log anomaly characteristics for each system hazard situation, includes:
Counting based on system hazard conditions existing in the system operation process, establishing a system hazard condition set, and establishing an abnormal log subset aiming at log expression of each system hazard condition;
according to records of system hazard conditions corresponding to the abnormality logs of the multiple devices which occur simultaneously in the past, the abnormality logs of the multiple devices are fed into corresponding abnormality log subsets;
scanning and analyzing the abnormal logs of the multiple devices in all the abnormal log subsets, determining the generating devices, generating modules, generating nodes and abnormal forms of the abnormal logs which occur simultaneously, combining the generating devices, the generating modules, the generating nodes and the abnormal forms of the abnormal logs, and determining the abnormal characteristics of the multiple device logs;
The method for establishing the association between the system hazard condition and the multi-device log abnormal characteristic comprises the following steps:
scanning and analyzing all the abnormal log subsets;
If the abnormal characteristics of the multi-equipment logs in different abnormal log subsets are the same, analyzing the abnormal characteristics of the multi-equipment logs with the same condition, determining the initiation probability between the abnormal characteristics of the multi-equipment logs with the same condition and the corresponding system hazard conditions, and establishing the association between the abnormal characteristics of the multi-equipment logs with the same condition and the system hazard conditions with the highest initiation probability;
For the multi-equipment log abnormal characteristics only existing in one abnormal log subset, directly establishing the association of the multi-equipment log abnormal characteristics and the system hazard conditions according to the association of the abnormal log subset and the corresponding system hazard conditions;
the method for determining the initiation probability between the abnormal characteristics of the multi-equipment log with the same condition and the corresponding system hazard condition comprises the following steps: according to records of system hazard conditions corresponding to past multi-equipment log exception features, the number of times of different system hazard conditions corresponding to the multi-equipment log exception features is obtained, and under the condition that the system is represented as the same multi-equipment log exception feature, the ratio of the number of times of occurrence of a certain system hazard condition to the total number of times of occurrence of the system hazard condition is determined as the initiation probability of the multi-equipment log exception feature.
In some embodiments of the present application, determining a number of log information pools requiring scan analysis based on the multi-device log anomaly characteristic determined at system runtime includes:
Scanning and analyzing the abnormal characteristics of the multi-equipment logs determined during system operation, determining an abnormal log type, and determining the log information pool with the same type as the abnormal log type aiming at the abnormal log type;
the method for scanning and analyzing the abnormal characteristics of the multi-device log comprises the following steps:
determining a log to be monitored based on the multi-equipment log monitoring directory table;
Establishing an abnormal characteristic judgment rule for each log;
based on the abnormal characteristic judging rule, scanning and analyzing the content of the log to be monitored, and determining the abnormal characteristics of the multi-equipment log;
In some embodiments of the present application, determining the current system hazard condition based on the coincidence of the first tag associated with the important log information in the log information pool with the multi-device log anomaly characteristic includes:
Scanning and analyzing the first tag groups of all important log information in the determined log information pools, and comparing and analyzing first tags in the first tag groups in different log information pools with the multi-equipment log abnormal characteristics;
If the log features combined by a plurality of first labels are consistent with one multi-equipment log abnormal feature, carrying out search analysis on the multi-equipment log abnormal feature in the system hazard set to determine the related system hazard condition;
In some embodiments of the present application, in order to alarm the running condition of the system in time, the audit method is improved, and the method further includes:
Determining a problem log record corresponding to the past major system operation problem based on the record of the past major system operation problem;
determining a log type contained in the issue log record based on the issue log record;
Carrying out statistical analysis on all the log types contained in the problem log records, sequencing the times of the log types, and carrying out monitoring weight division on the corresponding log types according to the sequence of the log types;
Determining a criticality factor of a log information pool of a corresponding type according to the monitoring weight division of the log type;
Determining a plurality of log information pools needing scanning analysis and a key factor corresponding to the log information pools based on the multi-equipment log abnormal characteristics, and determining a hazard degree value of the multi-equipment log abnormal characteristics;
performing hierarchical alarm according to the hazard degree value of the abnormal characteristics of the multi-equipment log;
the method for calculating the hazard value of the abnormal characteristics of the multi-equipment log comprises the following steps:
wherein Y is the harm degree value of the abnormal characteristics of the multi-equipment log, And the criticality factor i is the representative sequence number of the log information pool.
In some embodiments of the present application, in order to adjust the security performance of the system, the audit method is improved, and further includes:
matching the hazard value of the abnormal characteristics of the multi-equipment log with the determined hazard condition of the system, and determining the judging accuracy of the hazard condition of the system;
if the system hazard condition is judged to be wrong, the system hazard condition is redetermined;
the method for matching the hazard value of the multi-equipment log abnormal characteristic with the hazard condition of the system comprises the following steps:
setting a specific range of hazard values for each of the system hazard conditions;
Comparing and analyzing the hazard value of the abnormal characteristics of the multi-equipment log with the hazard value specific range of the hazard condition of the system, if the hazard value falls into the hazard value specific range, determining that the hazard condition of the system is accurate, and if the hazard value does not fall into the hazard value specific range, determining that the hazard condition of the system is incorrect.
In some embodiments of the present application, a method of redefining a system hazard condition is disclosed, the method of redefining a system hazard condition comprising:
Determining the initiation probability between the abnormal characteristics of the multi-equipment logs and different system hazard conditions based on the records of the system hazard conditions corresponding to the multi-equipment abnormal logs which occur simultaneously in the past;
Based on the abnormal characteristics of the multi-equipment logs and the initiation probability among different system hazard conditions, priority ordering is carried out on the different system hazard conditions to form a system hazard condition array { A1, A2, A3, … and An }, wherein the A1 element is a first system hazard condition, the A2 element is a second system hazard condition, the A3 element is a third system hazard condition and the An element is An nth system hazard condition, and priority orders of the first system hazard condition A1, the second system hazard condition A2, the third system hazard condition A3 and the nth system hazard condition An are gradually increased;
And when the hazard value corresponding to the abnormal characteristics of the multi-equipment log is gradually compared with the hazard value specific range corresponding to the elements in the system hazard condition arrays { A1, A2, A3, … and An }, if the hazard value corresponding to the abnormal characteristics of the multi-equipment log exists in the hazard value specific range of one element, the element is taken as the system hazard condition.
In some embodiments of the present application, there is also disclosed a multi-type device log audit system comprising: the system comprises a log monitoring module, an information extraction module, a classification module and a system hazard condition analysis module.
The log monitoring module is internally provided with a multi-device log monitoring list, and carries out log monitoring on corresponding devices based on the multi-device log monitoring list;
The information extraction module is used for carrying out scanning analysis on the monitored log, determining important log information based on a built-in important log characteristic judgment rule, and associating a first tag group with the important log information, wherein the first tag group comprises a plurality of first tags;
The classifying module is used for carrying out type analysis on the important log information and storing the important log information in different log information pools according to the types;
The system hazard condition analysis module is used for carrying out scanning analysis on multi-equipment log abnormal characteristics generated during system operation, determining a plurality of log information pools needing scanning analysis, and determining the current system hazard condition based on the condition that a first label associated with important log information in the log information pools is consistent with the multi-equipment log abnormal characteristics.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
Fig. 1 is a method step diagram of a log audit method for multiple types of devices according to an embodiment of the present application.
Detailed Description
The technical scheme of the invention is further described below through the attached drawings and the embodiments.
The technical solution of the present application will be clearly and completely described below with reference to the accompanying drawings and specific embodiments, it being understood that the preferred embodiments described herein are for illustrating and explaining the present application only and are not to be construed as limiting the scope of the present application, and that some insubstantial modifications and adaptations can be made by those skilled in the art in light of the following disclosure. In the present application, unless explicitly specified and defined otherwise, technical terms used in the present application should be construed in a general sense as understood by those skilled in the art to which the present application pertains. The terms "connected," "fixedly," "disposed" and the like are to be construed broadly and may be fixedly connected, detachably connected or integrally formed; can be directly connected or indirectly connected through an intermediate medium; either mechanically or electrically. Unless explicitly defined otherwise. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art according to the specific circumstances. Unless expressly stated or limited otherwise, a first feature "up" or "down" a second feature may be the first and second features in direct contact, or the first and second features in indirect contact through an intervening medium. Moreover, a first feature being "above" or "over" or "upper" a second feature may be a first feature being directly above or diagonally above the second feature, or simply indicating that the first feature is higher in level than the second feature. The first feature being "under" or "beneath" or "under" the second feature may be the first feature being directly under or obliquely under the second feature, or simply indicating that the first feature is level less than the second feature. Relational terms such as first, second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
Examples:
The invention aims to provide a log auditing method of multi-type equipment, which referring to fig. 1, comprises the following steps:
Step 1, constructing a multi-device log monitoring list, and carrying out log monitoring on corresponding devices based on the multi-device log monitoring list.
The multi-device log monitoring directory table can be established by a worker monitoring a corresponding operation module according to a system operation mechanism.
And 2, scanning and analyzing the monitored content of the log, determining important log information based on a preset important log characteristic judgment rule, and associating a first tag group with the important log information based on the content of the important log information, wherein the first tag group comprises a plurality of first tags.
The important log feature judging rule is a preset judging rule, and determines important log information segments of the operation condition of the relation system through analysis of past logs, specifically, identification codes are arranged on the log information before and after different information content segments, so that the important log feature judging rule can be a preset identification code, and determines important log information in the log information, such as a section of log information aaabbbccc, according to the identification codes, wherein aaa and ccc are identification codes, and when the identification codes are scanned in the process of scanning the log information, bbb is determined as important log information.
And 3, establishing a log information pool aiming at different types of log information, determining a log information pool to which the important log information belongs based on type analysis of the important log information, and classifying the important log information.
The log information pool is specifically storage space divided in the device, and each storage space corresponds to a corresponding type division.
The log types may include:
system log (system logs): and recording the running condition and the occurrence events of the operating system, including startup, shutdown, kernel crash, equipment failure and the like.
Application logs (application logs): the running condition of the application program and the occurred events are recorded, including errors, warnings, information and the like.
Security logs (security logs): system security events and security related operations such as login, access control, rights modification, etc. are recorded.
Access log (access logs): and recording access conditions of users or processes to resources in the system, such as website access logs, database access logs and the like.
Operation logs: and recording operation and maintenance processes and operation records of the system, including backup, recovery, upgrading, maintenance and the like.
Performance logs (performance logs): and recording the performance index and the running condition of the system, including indexes of CPU, memory, network and the like.
Application tracking log (application tracing logs): the key events and call links in the application are recorded for analysis and optimization of application performance.
And step 4, carrying out statistical analysis on each system hazard condition, and determining multi-equipment log abnormal characteristics of each system hazard condition.
The statistical analysis of the system hazard condition can be performed according to statistics of the system hazard condition of workers in the past operation process of the system, comparison analysis is performed on operation logs of different modules of different past equipment according to the same time node, abnormal features in the operation logs are extracted, specifically, analysis is performed on features different from usual features in the operation logs, for example, performance loads in one operation log are 30% -50% under normal conditions, if the load value breaks through 50% or is lower than 30%, abnormal conditions are determined, and information displayed by the break through 50% load and the load lower than 30% in the operation log is one element in the abnormal features of the multi-equipment logs.
And 5, determining a plurality of log information pools needing scanning analysis based on the multi-equipment log abnormal characteristics determined during system operation, and determining the current system hazard condition based on the coincidence condition of a first label associated with important log information in the log information pools and the multi-equipment log abnormal characteristics.
The log type corresponding to the abnormal characteristics can be determined in the multi-equipment log abnormal characteristics, the log information pools of the corresponding types are determined according to the determined log types, the performance consumption of monitoring and analyzing the log by the system is reduced, and the current system hazard condition is determined according to the first label of important log information.
In some embodiments of the present application, constructing a multi-device log monitoring directory table, and performing log monitoring on a corresponding device based on the multi-device log monitoring directory table, including:
the first step is to build a list of system application devices and determine the functions to be executed by the application devices according to the system operation requirements.
And secondly, determining a log to be monitored based on the function required to be executed by the application equipment, and generating an equipment log monitoring list based on the corresponding situation of the log and the application equipment.
In some embodiments of the present application, a scan analysis is performed on a monitored content of a log, and important log information is determined based on a preset important log feature judgment rule, and a first tag group is associated with the important log information based on the content of the important log information, where the first tag group includes a plurality of first tags, and includes:
step one, gradually scanning the logs, intercepting the logs conforming to the important log characteristic judgment rules, and generating important log information.
And secondly, extracting dynamic indexes in the important log information content and generating an index type first label.
And thirdly, extracting the occurrence time in the important log information content and generating a time-class first label.
And step four, extracting the equipment marks in the important log information content and generating a first label of the equipment class.
And fifthly, extracting the function module marks in the important log information content, and generating a function class first label.
Sixth, combining the index type first tag, the time type first tag, the equipment type first tag and the function type first tag to generate a first tag group.
And seventh, establishing an association relation between the first tag group and the important log information.
The method for extracting the dynamic index, the occurrence time, the equipment mark and the functional module mark in the important log information content comprises the following steps:
and establishing a log content identification table aiming at the dynamic index, the occurrence time, the equipment mark and the feature identification code of the function module mark in each log.
And carrying out scanning analysis on the important log information based on the log content identification table, and extracting information of dynamic indexes, occurrence time, equipment marks and functional module marks.
In some embodiments of the present application, performing a statistical analysis on each system hazard situation, determining multi-device log anomaly characteristics for each system hazard situation, includes:
the first step is to count based on the system hazard conditions existing in the running process of the system, establish a system hazard condition set and establish an abnormal log subset aiming at the log expression of each system hazard condition.
And secondly, according to the records of the system hazard conditions corresponding to the abnormality logs of the multiple devices which occur simultaneously in the past, the abnormality logs of the multiple devices are fed into the corresponding abnormality log subsets.
Thirdly, scanning and analyzing the abnormal logs of the multiple devices in all abnormal log subsets, determining the generating devices, generating modules, generating nodes and abnormal forms of the abnormal logs which occur simultaneously, and combining the generating devices, the generating modules, the generating nodes and the abnormal forms of the abnormal logs to determine the abnormal characteristics of the multiple device logs.
The method for establishing the association between the system hazard condition and the multi-device log abnormal characteristic comprises the following steps:
first, scanning and analyzing all the abnormal log subsets.
If the abnormal characteristics of the multi-equipment logs in different abnormal log subsets are the same, analyzing the abnormal characteristics of the multi-equipment logs in the same condition, determining the initiation probability between the abnormal characteristics of the multi-equipment logs in the same condition and the corresponding system hazard conditions, and establishing the association between the abnormal characteristics of the multi-equipment logs in the same condition and the system hazard conditions with the highest initiation probability.
It should be understood that the abnormal characteristics of the log of the multiple devices may correspond to multiple system hazard situations, so that separate analysis is required to be performed on the abnormal characteristics of the log of the multiple devices, so as to determine the system hazard situation with the highest probability of triggering the abnormal characteristics of the log of the multiple devices.
Thirdly, for the multi-equipment log abnormal characteristics only existing in one abnormal log subset, directly establishing the association between the multi-equipment log abnormal characteristics and the system hazard conditions according to the association between the abnormal log subset and the corresponding system hazard conditions.
The method for determining the initiation probability between the abnormal characteristics of the multi-equipment log with the same condition and the corresponding system hazard condition comprises the following steps: according to records of system hazard conditions corresponding to past multi-equipment log exception features, the number of times of different system hazard conditions corresponding to the multi-equipment log exception features is obtained, and under the condition that the system is represented as the same multi-equipment log exception feature, the ratio of the number of times of occurrence of a certain system hazard condition to the total number of times of occurrence of the system hazard condition is determined as the initiation probability of the multi-equipment log exception feature.
In some embodiments of the present application, determining a number of log information pools requiring scan analysis based on the multi-device log anomaly characteristic determined at system runtime includes:
and scanning and analyzing the abnormal characteristics of the multi-equipment logs determined during system operation, determining an abnormal log type, and determining the log information pool with the same type as the abnormal log type aiming at the abnormal log type.
The method for scanning and analyzing the abnormal characteristics of the multi-device log comprises the following steps:
And firstly, determining a log to be monitored based on the multi-device log monitoring directory table.
And secondly, establishing an abnormal characteristic judgment rule for each log.
And thirdly, based on the abnormal characteristic judgment rule, scanning and analyzing the content of the log to be monitored, and determining the abnormal characteristics of the multi-equipment log.
In some embodiments of the present application, determining the current system hazard condition based on the coincidence of the first tag associated with the important log information in the log information pool with the multi-device log anomaly characteristic includes:
the first step is to scan and analyze the first label group of all important log information in the determined log information pool, and to compare and analyze the first labels in the first label group in different log information pools with the abnormal characteristics of the multi-equipment log.
And secondly, if the log features combined by the plurality of first labels are consistent with one multi-equipment log abnormal feature, searching and analyzing the multi-equipment log abnormal feature in the system hazard set to determine the related system hazard condition.
In some embodiments of the present application, in order to alarm the running condition of the system in time, the audit method is improved, and the method further includes:
First, determining a problem log record corresponding to a past major system operation problem based on the record of the past major system operation problem.
And a second step of determining the log type contained in the problem log record based on the problem log record.
Thirdly, carrying out statistical analysis on all the log types contained in the problem log records, sequencing the times of the log types, and carrying out monitoring weight division on the corresponding log types according to the sequence of the log types.
And fourthly, determining the criticality factor of the log information pool of the corresponding type according to the monitoring weight division of the log type.
And fifthly, determining a plurality of log information pools needing scanning analysis and a key factor corresponding to the log information pools based on the multi-equipment log abnormal characteristics, and determining the hazard degree value of the multi-equipment log abnormal characteristics.
And step six, carrying out hierarchical alarm according to the hazard degree value of the abnormal characteristics of the multi-equipment log.
The method for calculating the hazard value of the abnormal characteristics of the multi-equipment log comprises the following steps:
wherein Y is the harm degree value of the abnormal characteristics of the multi-equipment log, And the criticality factor i is the representative sequence number of the log information pool.
In some embodiments of the present application, in order to adjust the security performance of the system, the audit method is improved, and further includes:
the first step, the hazard value of the abnormal characteristics of the multi-equipment log is matched with the determined hazard condition of the system, and the judgment accuracy of the hazard condition of the system is determined.
And step two, if the system hazard condition is judged to be wrong, the system hazard condition is redetermined.
The method for matching the hazard value of the abnormal characteristics of the multi-equipment log with the hazard condition of the system comprises the following steps:
In a first step, a specific range of hazard values is set for each of the system hazard conditions.
And secondly, comparing and analyzing the hazard value of the abnormal characteristics of the multi-equipment logs with the hazard value specific range of the hazard condition of the system, if the hazard value falls into the hazard value specific range, determining that the hazard condition of the system is accurate, and if the hazard value does not fall into the hazard value specific range, determining that the hazard condition of the system is incorrect.
In some embodiments of the present application, a method of redefining a system hazard condition is disclosed, the method of redefining a system hazard condition comprising:
The first step, determining the initiation probability between the abnormal characteristics of the multi-equipment logs and different system hazard conditions based on the records of the system hazard conditions corresponding to the multi-equipment abnormal logs which occur simultaneously in the past.
Secondly, based on the initiation probability between the abnormal characteristics of the multi-equipment logs and different system hazard conditions, the different system hazard conditions are prioritized to form a system hazard condition array { A1, A2, A3, …, an }, wherein the A1 element is a first system hazard condition, the A2 element is a second system hazard condition, the A3 element is a third system hazard condition, and the An element is An nth system hazard condition, and the priority orders of the first system hazard condition A1, the second system hazard condition A2, the third system hazard condition A3 and the nth system hazard condition An are gradually increased.
And thirdly, comparing the hazard value corresponding to the abnormal characteristics of the multi-equipment log with the specific range of hazard values corresponding to the elements in the system hazard condition arrays { A1, A2, A3, …, an } step by step, and taking the element as the system hazard condition if the hazard value corresponding to the abnormal characteristics of the multi-equipment log falls into the specific range of hazard value of one element.
In some embodiments of the present application, there is also disclosed a multi-type device log audit system comprising: the system comprises a log monitoring module, an information extraction module, a classification module and a system hazard condition analysis module.
The log monitoring module is internally provided with a multi-device log monitoring list, and carries out log monitoring on corresponding devices based on the multi-device log monitoring list.
The information extraction module is used for carrying out scanning analysis on the monitored log, determining important log information based on a built-in important log characteristic judgment rule, and associating a first tag group with the important log information, wherein the first tag group comprises a plurality of first tags.
The classifying module is used for carrying out type analysis on the important log information and storing the important log information in different log information pools according to the types.
The system hazard condition analysis module is used for carrying out scanning analysis on multi-equipment log abnormal characteristics generated during system operation, determining a plurality of log information pools needing scanning analysis, and determining the current system hazard condition based on the condition that a first label associated with important log information in the log information pools is consistent with the multi-equipment log abnormal characteristics.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention and not for limiting it, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that: the technical scheme of the invention can be modified or replaced by the same, and the modified technical scheme cannot deviate from the spirit and scope of the technical scheme of the invention.
Claims (8)
1. A method for auditing a log of multiple types of devices, comprising:
step 1, constructing a multi-device log monitoring list, and carrying out log monitoring on corresponding devices based on the multi-device log monitoring list;
step 2, scanning and analyzing the monitored content of the log, determining important log information based on a preset important log characteristic judgment rule, and associating a first tag group to the important log information based on the content of the important log information, wherein the first tag group comprises a plurality of first tags;
Step 3, establishing a log information pool aiming at different types of log information, determining a log information pool to which the important log information belongs based on type analysis of the important log information, and classifying the important log information;
step 4, carrying out statistical analysis on each system hazard condition, and determining multi-equipment log abnormal characteristics of each system hazard condition;
Step 5, determining a plurality of log information pools needing scanning analysis based on the multi-equipment log abnormal characteristics determined during system operation, and determining the current system hazard condition based on the coincidence condition of a first label associated with important log information in the log information pools and the multi-equipment log abnormal characteristics;
Constructing a multi-device log monitoring list, and carrying out log monitoring on corresponding devices based on the multi-device log monitoring list, wherein the method comprises the following steps:
establishing a system application equipment list, and determining the functions to be executed by the application equipment according to the system operation requirements;
determining a log to be monitored based on a function to be executed by the application equipment, and generating an equipment log monitoring list based on the corresponding situation of the log and the application equipment;
scanning and analyzing the monitored content of the log, determining important log information based on a preset important log characteristic judgment rule, and associating a first tag group with the important log information based on the content of the important log information, wherein the first tag group comprises a plurality of first tags and comprises:
Gradually scanning the logs, intercepting the logs conforming to the important log characteristic judgment rules, and generating important log information;
extracting dynamic indexes in the important log information content and generating an index class first label;
extracting the occurrence time in the important log information content and generating a time-class first label;
extracting the equipment marks in the important log information content and generating a first equipment type label;
Extracting the function module marks in the important log information content and generating a function class first label;
combining the index type first tag, the time type first tag, the equipment type first tag and the function type first tag to generate a first tag group;
Establishing an association relation between the first tag group and the important log information;
The method for extracting the dynamic index, the occurrence time, the equipment mark and the functional module mark in the important log information content comprises the following steps:
Establishing a log content identification table aiming at the dynamic index, the occurrence time, the equipment mark and the feature identification code of the function module mark in each log;
and carrying out scanning analysis on the important log information based on the log content identification table, and extracting information of dynamic indexes, occurrence time, equipment marks and functional module marks.
2. The method of claim 1, wherein the step of statistically analyzing each system hazard condition to determine a multi-device log anomaly characteristic for each system hazard condition comprises:
Counting based on system hazard conditions existing in the system operation process, establishing a system hazard condition set, and establishing an abnormal log subset aiming at log expression of each system hazard condition;
according to records of system hazard conditions corresponding to the abnormality logs of the multiple devices which occur simultaneously in the past, the abnormality logs of the multiple devices are fed into corresponding abnormality log subsets;
scanning and analyzing the abnormal logs of the multiple devices in all the abnormal log subsets, determining the generating devices, generating modules, generating nodes and abnormal forms of the abnormal logs which occur simultaneously, combining the generating devices, the generating modules, the generating nodes and the abnormal forms of the abnormal logs, and determining the abnormal characteristics of the multiple device logs;
The method for establishing the association between the system hazard condition and the multi-device log abnormal characteristic comprises the following steps:
scanning and analyzing all the abnormal log subsets;
If the abnormal characteristics of the multi-equipment logs in different abnormal log subsets are the same, analyzing the abnormal characteristics of the multi-equipment logs with the same condition, determining the initiation probability between the abnormal characteristics of the multi-equipment logs with the same condition and the corresponding system hazard conditions, and establishing the association between the abnormal characteristics of the multi-equipment logs with the same condition and the system hazard conditions with the highest initiation probability;
For the multi-equipment log abnormal characteristics only existing in one abnormal log subset, directly establishing the association of the multi-equipment log abnormal characteristics and the system hazard conditions according to the association of the abnormal log subset and the corresponding system hazard conditions;
the method for determining the initiation probability between the abnormal characteristics of the multi-equipment log with the same condition and the corresponding system hazard condition comprises the following steps:
According to records of system hazard conditions corresponding to past multi-equipment log exception features, the number of times of different system hazard conditions corresponding to the multi-equipment log exception features is obtained, and under the condition that the system is represented as the same multi-equipment log exception feature, the ratio of the number of times of occurrence of a certain system hazard condition to the total number of times of occurrence of the system hazard condition is determined as the initiation probability of the multi-equipment log exception feature.
3. The method for auditing logs of multiple types of devices according to claim 2, wherein determining a plurality of log information pools to be scanned and analyzed based on the abnormal characteristics of the logs of multiple types of devices determined during system operation comprises:
Scanning and analyzing the abnormal characteristics of the multi-equipment logs determined during system operation, determining an abnormal log type, and determining the log information pool with the same type as the abnormal log type aiming at the abnormal log type;
the method for scanning and analyzing the abnormal characteristics of the multi-device log comprises the following steps:
determining a log to be monitored based on the multi-equipment log monitoring directory table;
Establishing an abnormal characteristic judgment rule for each log;
And based on the abnormal characteristic judging rule, scanning and analyzing the content of the log to be monitored, and determining the abnormal characteristics of the multi-equipment log.
4. The method of claim 2, wherein determining the current system hazard condition based on the coincidence of the first tag associated with the important log information in the log information pool with the multi-device log anomaly characteristic comprises:
Scanning and analyzing the first tag groups of all important log information in the determined log information pools, and comparing and analyzing first tags in the first tag groups in different log information pools with the multi-equipment log abnormal characteristics;
If the log features combined by the plurality of first labels are consistent with one multi-equipment log abnormal feature, the multi-equipment log abnormal feature is searched and analyzed in the system hazard set, and the related system hazard condition is determined.
5. The multi-type device log auditing method of claim 1, further comprising:
Determining a problem log record corresponding to the past major system operation problem based on the record of the past major system operation problem;
determining a log type contained in the issue log record based on the issue log record;
Carrying out statistical analysis on all the log types contained in the problem log records, sequencing the times of the log types, and carrying out monitoring weight division on the corresponding log types according to the sequence of the log types;
Determining a criticality factor of a log information pool of a corresponding type according to the monitoring weight division of the log type;
Determining a plurality of log information pools needing scanning analysis and a key factor corresponding to the log information pools based on the multi-equipment log abnormal characteristics, and determining a hazard degree value of the multi-equipment log abnormal characteristics;
performing hierarchical alarm according to the hazard degree value of the abnormal characteristics of the multi-equipment log;
the method for calculating the hazard value of the abnormal characteristics of the multi-equipment log comprises the following steps:
wherein Y is the harm degree value of the abnormal characteristics of the multi-equipment log, And the criticality factor i is the representative sequence number of the log information pool.
6. The multi-type device log auditing method of claim 5, further comprising:
matching the hazard value of the abnormal characteristics of the multi-equipment log with the determined hazard condition of the system, and determining the judging accuracy of the hazard condition of the system;
if the system hazard condition is judged to be wrong, the system hazard condition is redetermined;
The method for matching the hazard value of the abnormal characteristics of the multi-equipment log with the hazard condition of the system comprises the following steps:
setting a specific range of hazard values for each of the system hazard conditions;
Comparing and analyzing the hazard value of the abnormal characteristics of the multi-equipment log with the hazard value specific range of the hazard condition of the system, if the hazard value falls into the hazard value specific range, determining that the hazard condition of the system is accurate, and if the hazard value does not fall into the hazard value specific range, determining that the hazard condition of the system is incorrect.
7. The method of claim 6, wherein the method of redefining system hazard conditions comprises:
Determining the initiation probability between the abnormal characteristics of the multi-equipment logs and different system hazard conditions based on the records of the system hazard conditions corresponding to the multi-equipment abnormal logs which occur simultaneously in the past;
Based on the abnormal characteristics of the multi-equipment logs and the initiation probability among different system hazard conditions, priority ordering is carried out on the different system hazard conditions to form a system hazard condition array { A1, A2, A3, … and An }, wherein the A1 element is a first system hazard condition, the A2 element is a second system hazard condition, the A3 element is a third system hazard condition and the An element is An nth system hazard condition, and priority orders of the first system hazard condition A1, the second system hazard condition A2, the third system hazard condition A3 and the nth system hazard condition An are gradually increased;
And when the hazard value corresponding to the abnormal characteristics of the multi-equipment log is gradually compared with the hazard value specific range corresponding to the elements in the system hazard condition arrays { A1, A2, A3, … and An }, if the hazard value corresponding to the abnormal characteristics of the multi-equipment log exists in the hazard value specific range of one element, the element is taken as the system hazard condition.
8. A multi-type device log audit system comprising:
The log monitoring module is internally provided with a multi-device log monitoring list and carries out log monitoring on corresponding devices based on the multi-device log monitoring list;
the information extraction module is used for carrying out scanning analysis on the monitored log, determining important log information based on a built-in important log characteristic judgment rule, and associating a first tag group with the important log information, wherein the first tag group comprises a plurality of first tags;
The classification module is used for carrying out type analysis on the important log information and storing the important log information in different log information pools according to the types;
The system hazard condition analysis module is used for carrying out scanning analysis on the abnormal characteristics of the multi-equipment log generated during system operation, determining a plurality of log information pools needing scanning analysis, and determining the current system hazard condition based on the coincidence condition of a first label associated with important log information in the log information pools and the abnormal characteristics of the multi-equipment log;
Constructing a multi-device log monitoring list, and carrying out log monitoring on corresponding devices based on the multi-device log monitoring list, wherein the method comprises the following steps:
establishing a system application equipment list, and determining the functions to be executed by the application equipment according to the system operation requirements;
determining a log to be monitored based on a function to be executed by the application equipment, and generating an equipment log monitoring list based on the corresponding situation of the log and the application equipment;
scanning and analyzing the monitored content of the log, determining important log information based on a preset important log characteristic judgment rule, and associating a first tag group with the important log information based on the content of the important log information, wherein the first tag group comprises a plurality of first tags and comprises:
Gradually scanning the logs, intercepting the logs conforming to the important log characteristic judgment rules, and generating important log information;
extracting dynamic indexes in the important log information content and generating an index class first label;
extracting the occurrence time in the important log information content and generating a time-class first label;
extracting the equipment marks in the important log information content and generating a first equipment type label;
Extracting the function module marks in the important log information content and generating a function class first label;
combining the index type first tag, the time type first tag, the equipment type first tag and the function type first tag to generate a first tag group;
Establishing an association relation between the first tag group and the important log information;
The method for extracting the dynamic index, the occurrence time, the equipment mark and the functional module mark in the important log information content comprises the following steps:
Establishing a log content identification table aiming at the dynamic index, the occurrence time, the equipment mark and the feature identification code of the function module mark in each log;
and carrying out scanning analysis on the important log information based on the log content identification table, and extracting information of dynamic indexes, occurrence time, equipment marks and functional module marks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310385033.1A CN116610642B (en) | 2023-04-11 | 2023-04-11 | Log auditing method and system for multiple types of equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310385033.1A CN116610642B (en) | 2023-04-11 | 2023-04-11 | Log auditing method and system for multiple types of equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116610642A CN116610642A (en) | 2023-08-18 |
| CN116610642B true CN116610642B (en) | 2024-05-28 |
Family
ID=87680754
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310385033.1A Active CN116610642B (en) | 2023-04-11 | 2023-04-11 | Log auditing method and system for multiple types of equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116610642B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111078455A (en) * | 2019-12-24 | 2020-04-28 | 北京优特捷信息技术有限公司 | Abnormal behavior sequence correlation processing method and device based on time axis, equipment and storage medium |
| CN113391990A (en) * | 2021-06-30 | 2021-09-14 | 未鲲(上海)科技服务有限公司 | System log monitoring method, device, equipment and storage medium |
| CN113496032A (en) * | 2020-04-03 | 2021-10-12 | 中国信息安全测评中心 | Big data operation abnormity monitoring system based on distributed computation and rule engine |
| CN115622873A (en) * | 2022-09-05 | 2023-01-17 | 四川华能嘉陵江水电有限责任公司 | Comprehensive log analysis system |
-
2023
- 2023-04-11 CN CN202310385033.1A patent/CN116610642B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111078455A (en) * | 2019-12-24 | 2020-04-28 | 北京优特捷信息技术有限公司 | Abnormal behavior sequence correlation processing method and device based on time axis, equipment and storage medium |
| CN113496032A (en) * | 2020-04-03 | 2021-10-12 | 中国信息安全测评中心 | Big data operation abnormity monitoring system based on distributed computation and rule engine |
| CN113391990A (en) * | 2021-06-30 | 2021-09-14 | 未鲲(上海)科技服务有限公司 | System log monitoring method, device, equipment and storage medium |
| CN115622873A (en) * | 2022-09-05 | 2023-01-17 | 四川华能嘉陵江水电有限责任公司 | Comprehensive log analysis system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116610642A (en) | 2023-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4318643B2 (en) | Operation management method, operation management apparatus, and operation management program | |
| CN112308126B (en) | Fault identification model training method, fault identification method, device and electronic equipment | |
| CN109669844B (en) | Equipment fault processing method, device, equipment and storage medium | |
| CN113553210A (en) | Alarm data processing method, device, equipment and storage medium | |
| EP3663919B1 (en) | System and method of automated fault correction in a network environment | |
| US10248517B2 (en) | Computer-implemented method, information processing device, and recording medium | |
| CN113254255B (en) | Cloud platform log analysis method, system, device and medium | |
| US11954945B2 (en) | Systems and methods for analyzing machine performance | |
| CN111400435B (en) | Mail alarm convergence method, device, computer equipment and storage medium | |
| CN116610642B (en) | Log auditing method and system for multiple types of equipment | |
| KR101810860B1 (en) | Integrated monitoring system for personal information security product | |
| CN112363891B (en) | Method for obtaining abnormal reasons based on fine-grained events and KPIs (Key Performance indicators) analysis | |
| CN111309584A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN112737120B (en) | Regional power grid control report generation method and device and computer equipment | |
| CN112699005A (en) | Server hardware fault monitoring method, electronic equipment and storage medium | |
| CN118520517B (en) | Solid state disk data protection system based on error check | |
| KR102523671B1 (en) | Log-based anomaly detection system of autonomous driving system and its operation method | |
| CN113806196B (en) | Root cause analysis method and system | |
| CN118730337B (en) | Three-dimensional temperature early warning method and system for high-voltage equipment | |
| CN118138270A (en) | Method and system for security detection of server log process | |
| CN116860578B (en) | Network and information security log management system and method | |
| US7292905B1 (en) | Method and system for identifying manufacturing anomalies in a manufacturing system | |
| CN114254330A (en) | Industrial network security identification method and system | |
| CN114722390A (en) | Method, device, equipment and medium for safety data integration and feature extraction | |
| CN115269306A (en) | Data compression method and system based on machine learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |