TW201822018A - Smart monitoring and early warning device for distributed software defined storage system and method thereof wherein the method includes gradually adjusting configuration based on an abnormal comparison result - Google Patents

Abstract

The invention discloses a smart monitoring and early warning device for a distributed software defined storage system and a method thereof. The method includes utilizing a state data collection module to collect state data of each node in operation of the distributed software defined storage system, and utilizing an intelligent analysis module to receive and analyze the state data. The method further includes comparing the state data with an abnormal model data set to generate an abnormal comparison result data set, and utilizing an early warning and response module to read the current configuration data of the distributed software defined storage system and calculating a target configuration data according to the abnormal level after receiving the abnormal comparison result data set. The method also includes utilizing the early warning and response module to calculate a difference between the current configuration data and the target configuration data and gradually adjusting the configurations of the distributed software defined storage system.

Description

   Intelligent monitoring and early warning device and method applied to distributed software-defined storage system   

The invention relates to an intelligent monitoring and early warning device and method applied to a distributed software-defined storage system, so as to reduce the maintenance cost of the distributed software-defined storage system and improve service quality through an automated monitoring and response process. .

The use of storage systems has received great attention in the current data center environment, because whether it is a new system construction demand caused by a pioneering and innovative business type, or the introduction of long-established and mature IT applications , All need storage systems to store data or use for analysis; and based on the increasing capacity of the system, the existing distributed storage system can cut and store data, allowing users to use parallel technology to accelerate data calculations. , And through the backup mechanism to improve the fault tolerance of data, so that large data storage is no longer an issue.

In order to ensure the service quality of the storage system and solve the shortcomings of the traditional storage system architecture, such as the centralized management of storage resources is not easy, etc., a software-defined storage (SDS) system has been developed in the market. Software-defined storage is an evolutionary concept of computer data storage. It uses software-controlled methods to determine data storage policies and management methods. It can separate computer hardware data storage technology from the software that manages the storage infrastructure. Under software-defined storage, you can activate features such as deduplication, replication, thin provisioning, snapshots, and backups, and provide policy management of storage resources.

Among them, software-defined storage can be combined with distributed storage systems to become a distributed software-defined storage system; distributed software-defined storage systems can use software to handle data protection, allowing software to achieve higher levels of protection more flexibly, allowing more If multiple drives fail at the same time, it will not cause data loss, and the distributed software-defined storage system also has the function of performance expansion and self-repair mechanism. However, although the distributed software-defined storage system has many advantages, there is no mechanism to prevent abnormality in the distributed software-defined storage system at the current stage. Even if it has a self-repairing function, it still cannot stop the abnormality in real time. The impact of this will affect the stability of the storage system performance.

In view of the problems of the above-mentioned conventional techniques, an object of the present invention is to provide an intelligent monitoring and early warning device and method applied to a distributed software-defined storage system, so as to reduce the distributed software definition through an automated monitoring and reaction process Maintenance costs of storage systems and improved service quality.

According to the purpose of the present invention, an intelligent monitoring and early warning device applied to a distributed software-defined storage system is provided. The intelligent monitoring and early-warning device includes: a state data collection module, which collects state data of each node in the distributed software-defined storage system. ; A smart analysis module is connected to the status data collection module to receive the status data and analyze it, and the smart analysis module further compares the status data with an abnormal model data to generate an abnormal comparison result Data; and an early warning and response module, which is connected to the intelligent analysis module, said early warning and response module reads the current configuration data of one of the distributed software-defined storage systems, and receives the abnormal comparison result data after receiving A target configuration data is calculated according to the degree of abnormality, and the early warning and response module compares the difference between the current configuration data and the target configuration data, and then gradually adjusts the configuration of the distributed software-defined storage system in an asymptotic manner.

According to the purpose of the present invention, a smart monitoring and early warning method applied to a distributed software-defined storage system is further provided, which includes the following steps: a state data collection module is used to collect one of the operations of each node of the distributed software-defined storage system. Status data; use a smart analysis module to receive status data and analyze it, and further compare the status data with an abnormal model data to generate an abnormal comparison result data; use an early warning and response module to read the decentralized The current configuration data of one of the software-defined storage systems, and after receiving the abnormal comparison result data, calculates a target configuration data based on the abnormality degree; and uses an early warning and response module to compare the difference between the current configuration data and the target configuration data, Then gradually adjust the configuration of the distributed software-defined storage system in an asymptotic manner.

According to the above technical features, the present invention further includes a status database, which is connected to the status data collection module to store status data.

According to the above technical features, the smart analysis module is connected to the status database, and the smart analysis module reads the existing data in the status database and performs calculations and analysis to construct a comparison between the normal status data and the abnormal status data. The abnormal model data is output, and the intelligent analysis module receives analysis feedback data input by a user to update and adjust the abnormal model data.

According to the above technical characteristics, the status data includes processor usage, memory usage, disk access throughput, disk access operation rate, disk access response time, disk health information, and network usage traffic. And node response time.

According to the above technical characteristics, the adjustment configuration of the early warning and response module in an asymptotic manner is to perform a single adjustment within a specific time, and then perform the next adjustment after the data recovery status of the distributed software-defined storage system is stable.

In summary, the intelligent monitoring and early warning device and method thereof applied to the distributed software-defined storage system of the present invention have one or more of the following characteristics:

1. The present invention establishes an anomaly model through automated status data collection and analysis, and can instantly determine whether each monitored device or equipment has an abnormal tendency when the system is running, and then detect potential anomalies, and feedback by manual interpretation Correct the abnormal model to improve the accuracy of judgment.

2. The early warning and response module of the present invention uses the intelligent analysis module to analyze the abnormal situation, decides a new distributed software-defined storage system configuration, compares the current configuration, and gradually adjusts the configuration in a gradual manner, and controls the adjustment range to make the distributed The software-defined storage system can return to a stable state within a certain time, which can effectively avoid affecting the service quality of the distributed software-defined storage system.

3. Under the monitoring and reaction process of the present invention, maintenance personnel can prepare in advance for the expected abnormality and deal with it as soon as the abnormality occurs, making the maintenance operation more efficient. .

10‧‧‧Status data collection module

20‧‧‧Smart Analysis Module

21‧‧‧ Anomaly Model Data

30‧‧‧Early Warning and Response Module

40‧‧‧Status database

100‧‧‧ Distributed Software Defined Storage System

101‧‧‧node

S11 ~ S14‧‧‧step flow

S21 ~ S28‧‧‧step flow

S31 ~ S38‧‧‧step flow

FIG. 1 is a schematic diagram of the intelligent monitoring and early warning device of the present invention.

FIG. 2 is a flowchart of the intelligent monitoring and early warning method of the present invention.

FIG. 3 is a flowchart of analyzing status data by the intelligent analysis module of the present invention.

FIG. 4 is a flowchart of processing an abnormality analysis result by the early warning and response module of the present invention.

In order to help examiners understand the technical features, contents and advantages of the present invention and the effects that can be achieved, the present invention will be described in detail in conjunction with the accompanying drawings in the form of embodiments, and the drawings used therein, The main purpose is only for the purpose of illustration and supplementary description. It may not be the actual proportion and precise configuration after the implementation of the invention. Therefore, the attached drawings should not be interpreted and limited to the scope of rights of the present invention in actual implementation. He Xianming.

The invention mainly proposes an intelligent monitoring and early warning device and method applied to a distributed software-defined storage system, which can collect and store monitoring data and software operation records of each node in the distributed software-defined storage system, and then use data statistics , Anomaly detection, machine learning and other methods of real-time analysis, in order to find abnormal data that may cause obstacles when hardware obstacles occur, and create abnormal model data. If an abnormal data pattern is detected during the subsequent operation of the distributed software-defined storage system, an early warning can be issued for the abnormality, and the data storage proportion can be adjusted to enable the distributed software-defined storage system to move the data out of the abnormality area and make the abnormality earlier. In addition to reducing the impact on distributed software-defined storage system services when anomalies occur, preparation can also accelerate the process of replacing damaged hardware to maintain the stable performance of the distributed software-defined storage system.

In order to more clearly describe the technical features of the present invention, please refer to FIG. 1, which is a schematic diagram of the intelligent monitoring and early warning device of the present invention. The intelligent monitoring and early warning device applicable to the distributed software-defined storage system of the present invention mainly includes a state data collection module 10, a smart analysis module 20, an early warning and response module 30, and a state database 40, and the state data collection The module 10 is connected to the status database 40, and the smart analysis module 20 is connected to the status data collection module 10, the early warning and response module 30, and the status database 40.

Each node 101 in the monitored distributed software-defined storage system 100 deploys an agent program for collecting status data, the agent program periodically sends status data to the status data collection module 10, and the status data collection module After the group 10 receives the latest status data, it stores the status data in the status database 40 and sends it to the smart analysis module 20 for abnormality analysis. Among them, the status data includes processor usage, memory usage, disk access throughput, disk access operation rate, disk access response time, disk health information, network usage traffic, and node response. Time and other data.

When the intelligent analysis module 20 starts, it reads the existing status data in the status database 40 to construct abnormal model data 21, and when the intelligent analysis module 20 receives the latest status data transmitted by the status collection module 10 The analysis will be performed according to the abnormal model data 21, and then the abnormal comparison result data generated after the analysis is transmitted to the early warning and response module 30. In detail, the intelligent analysis module 20 reads the existing data in the state database 40 and performs calculations and analysis to compare the normal state data with the abnormal state data to construct the abnormal model data 21, and the intelligent analysis After receiving the status data transmitted by the status collection module 10, the module 20 can detect whether there is a potential abnormality. At this time, the status data is normalized and the abnormal data is filtered initially, and then the status data and the abnormality are further filtered. The model data 21 is compared to generate the abnormal comparison result data.

When the early warning and response module 30 runs, it will detect the current setting and configuration of the distributed software-defined storage system 100, and when the early warning and response module 30 receives the abnormal comparison result data transmitted by the intelligent analysis module 20, it will It will send early warning messages to maintenance personnel, and calculate a new configuration based on the abnormal comparison result data and compare the current configuration, and then gradually adjust the distributed software-defined storage system 100 in an asymptotic manner to maintain the stable state and provide services. . In detail, the early warning and response module 30 reads the current configuration data of the distributed software-defined storage system 100, and calculates a target configuration data based on the abnormality degree when receiving the abnormal comparison result data, and the early warning and response After comparing the difference between the current configuration data and the target configuration data, the module 30 will gradually adjust the configuration of the distributed software-defined storage system 100 in an asymptotic manner, and the asymptotic adjustment method is to wait for dispersion in each adjustment. After the data recovery status of the distributed software-defined storage system 100 is stable, the next adjustment is performed, and each adjustment is controlled to complete within a certain time, so as to ensure the operation and service quality of the distributed software-defined storage system 100.

In the above, the maintenance personnel can actually confirm the abnormal situation on the decentralized software-defined storage system 100, and return an analysis feedback data to the early warning and response module 30, and the early warning and response module 30 can return the analysis and feedback data It is sent to the intelligent analysis module 20, so that the intelligent analysis module 20 updates and adjusts the abnormal model data 21 and the status database 50 after receiving the analysis feedback data, so as to correct the subsequent analysis, thereby avoiding misjudgement.

Please refer to FIG. 2, which is a flowchart of the intelligent monitoring and early warning method of the present invention. The process steps are as follows:

Step S11: Use a status data collection module to collect status data of each node of the distributed software-defined storage system.

Step S12: Use a smart analysis module to receive and analyze the status data, and further compare the status data with an abnormal model data to generate an abnormal comparison result data.

Step S13: using an early warning and response module to read the current configuration data of one of the distributed software-defined storage systems, and after receiving the abnormal comparison result data, calculate a target configuration data according to the abnormality degree.

Step S14: Use the early warning and response module to compare the difference between the current configuration data and the target configuration data, and then gradually adjust the configuration of the distributed software-defined storage system in an asymptotic manner.

Please refer to FIG. 3 again, which is a flowchart of analyzing the status data by the intelligent analysis module of the present invention. The process steps are as follows: Step S21: Receive the status data message sent by the status collection module. Step S22: identify whether the status of the target monitored by the status data has been previously marked as abnormal, and if so, skip to step S28 to maintain and determine the abnormality, otherwise continue to the following steps. Step S23: Normalize according to different types of status data to facilitate subsequent analysis. Step S24: Determine whether the received status data is within the normal range counted in the abnormal model. If so, skip to step S27 to determine whether the monitored target is normal, otherwise continue to the following steps. Step S25: Calculate whether the duration of the status data is not within the normal range, whether it exceeds the tolerable observation period, and if so, skip to step S28 to determine whether the monitored target is abnormal, otherwise continue with the following steps. Step S26: Compare the abnormal model data of the monitoring target based on the status data, determine whether it meets the characteristics of the previous abnormality, and calculate the degree of similarity to indicate the possibility of abnormality. If it exceeds a certain value, the target of the monitoring is abnormal, otherwise it is normal. .

Please refer to FIG. 4 again, which is a flowchart of processing the abnormal analysis result by the early warning and response module of the present invention. The flow steps are as follows: Step S31: receiving abnormal comparison result data transmitted by the intelligent analysis module, and triggering step S32. , And it is determined in step S35 whether there is a new potential abnormality monitoring target in the result, and if so, step S36 is triggered together. Step S32: Calculate a new target configuration based on the abnormal comparison result data, including strategies such as data placement ratio. Step S33: Read the current configuration and compare it with the new target configuration to calculate the difference between them. Step S34: The configuration is gradually adjusted in an asymptotic manner. Each adjustment will be fine-tuned according to the time spent in the previous adjustment so that the adjustment time can be controlled within a certain range to maintain the stability of the storage system. Step S36: Issue a new potential abnormal alert to the maintenance personnel. Step S37: The maintenance personnel give feedback after confirming the actual situation, confirming the existence of the potential abnormality. Step S38: Send the feedback from the maintenance personnel to the intelligent analysis module, so as to facilitate the correction of the abnormal module data and subsequent judgment.

Specifically, the present invention is divided into three major modules, including a state data collection module, a smart analysis module, and an early warning and response module. The status data collection module is responsible for collecting the status data of each node in the distributed software-defined storage system and storing it in the status database; the smart analysis module is responsible for analyzing the status data, constructing an abnormal model, and determining the abnormality of each status data monitoring target ; The early warning and response module is responsible for reporting potential anomalies found and adjusting the storage system configuration based on the degree of anomaly. By using the present invention to assist the operation of the distributed software-defined storage system in an automated and intelligent manner, advance warning allows maintenance personnel to prepare or handle abnormal equipment in advance, and adjust the storage system configuration for potential abnormalities to avoid affecting performance due to abnormalities, which can greatly Reduce the cost of management and maintenance of distributed software-defined storage systems.

In view of the above, it can be seen that the present invention has indeed achieved the desired effect under the breakthrough of the previous technology, and it is not easy for those skilled in the art to think about it. Furthermore, the invention has not been disclosed before the application, and its features Progressiveness and practicability show that it has met the application requirements for patents, and submitted a patent application in accordance with the law. I urge your office to approve this application for an invention patent to encourage inventions and to be virtuous.

The above-mentioned embodiments are only for explaining the technical ideas and characteristics of the present invention. The purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. When the scope of the patent of the present invention cannot be limited, That is, any equivalent changes or modifications made in accordance with the spirit disclosed in the present invention should still be covered by the patent scope of the present invention.

Claims (10)

    An intelligent monitoring and early warning device applied to a distributed software-defined storage system includes: a state data collection module, which collects state data of each node operation of the distributed software-defined storage system; a smart analysis module, Is connected to the status data collection module to receive the status data and analyze it, and the smart analysis module further compares the status data with an abnormal model data to generate an abnormal comparison result data; and The early warning and response module is connected to the intelligent analysis module. The early warning and response module reads the current configuration data of one of the distributed software-defined storage systems and receives the abnormal comparison result data based on the degree of abnormality. A target configuration data is calculated, and the early warning and response module compares the difference between the current configuration data and the target configuration data, and then gradually adjusts the configuration of the distributed software-defined storage system in an asymptotic manner.         According to the intelligent monitoring and early warning device described in item 1 of the scope of patent application, it further includes a status database connected to the status data collection module to store the status data.         The intelligent monitoring and early warning device as described in the second item of the patent application scope, wherein the intelligent analysis module is connected to the status database, and the intelligent analysis module reads the existing data in the status database and performs calculations. And analyzing, comparing the normal state data with the abnormal state data to construct the abnormal model data, and the intelligent analysis module receives analysis feedback data input by the user to update and adjust the abnormal model data.         The intelligent monitoring and early warning device according to item 1 of the scope of patent application, wherein the status data includes processor usage, memory usage, disk access throughput, disk access operation rate, and disk storage Take response time, disk health information, network usage traffic, and node response time.         The intelligent monitoring and early warning device as described in the first item of the scope of patent application, wherein the early warning and response module is adjusted in an asymptotic manner. The adjustment is performed once within a specific time, and in a distributed software-defined storage system. After the data recovery status is stable, the next adjustment is performed.         An intelligent monitoring and early-warning method applied to a distributed software-defined storage system includes the following steps: using a state data collection module to collect state data of each node of the distributed software-defined storage system; using an intelligent analysis module The group receives the status data and analyzes it, and further compares the status data with an abnormal model data to generate an abnormal comparison result data; using an early warning and response module to read one of the distributed software-defined storage systems Current configuration data, and calculate a target configuration data according to the abnormality degree after receiving the abnormal comparison result data; and use the early warning and response module to compare the degree of difference between the current configuration data and the target configuration data, and then approach the Ways to gradually adjust the configuration of the distributed software-defined storage system.         As described in item 6 of the scope of the patent application, the intelligent monitoring and early warning method applied to a distributed software-defined storage system further includes the following steps: using a status database to store the status data.         As described in item 7 of the scope of the patent application, the intelligent monitoring and early warning method applied to a distributed software-defined storage system further includes the following steps: using the intelligent analysis module to read the existing data in the status database and perform Calculation and analysis, comparing the normal state data with the abnormal state data to construct the abnormal model data, and using the intelligent analysis module to receive the analysis feedback data input by the user to update and adjust the abnormal model data.         As described in item 6 of the scope of the patent application, the intelligent monitoring and early warning method applied to a distributed software-defined storage system, wherein the status data includes processor usage, memory usage, disk access throughput traffic, magnetic Disk access operation rate, disk access response time, disk health information, network usage traffic, and node response time.         The intelligent monitoring and early warning method applied to the decentralized software-defined storage system as described in item 6 of the scope of patent application, wherein the early warning and response module adjusts the configuration in an asymptotic manner to perform a single adjustment within a specific time And make the next adjustment after the data recovery status of the distributed software-defined storage system is stable.