CN117632666A - Alarm method, device and storage medium - Google Patents

Abstract

The embodiment of the application provides an alarm method, alarm equipment and a storage medium. Under the condition of an alarm triggering event, determining a request path for the abnormal IO requests, and clustering the abnormal IO requests based on the request path to generate at least one abnormal IO request group, wherein the abnormal IO requests corresponding to the request path passing through the same physical node are positioned in the same abnormal IO request group; moreover, it is proposed to output alarm information in units of abnormal IO request groups. In this way, from the perspective of a single physical node with an exception, the exception IO requests caused by the physical node can be clustered in the same exception IO request group because the request paths all pass through the physical node, so that the exception IO requests caused by the same exception cause can be ensured to realize alarming in one piece of alarming information, repeated alarming on the same exception cause is avoided, the number of the alarming information is reduced, the alarming information can be processed in time, and the processing efficiency of alarming is improved.

Description

Alarm method, device and storage medium

Technical field

This application relates to the field of cloud storage technology, and in particular to an alarm method, equipment and storage medium.

Background technique

Cloud storage can be understood as a mode of online online storage (Cloud storage). With the development of cloud storage technology, a storage and computing separation architecture has also been proposed based on cloud storage technology, referred to as storage and computing separation architecture. Cloud storage technology can be used to implement the storage layer in a storage-computing separation architecture. The storage and computing separation architecture can also include a computing layer. The computing layer and storage layer are decoupled and connected through the network. Both the computing layer and the storage layer can be implemented as independent distributed systems. Each computing node in the computing layer can access the storage layer through IO requests to read and write data from the storage nodes in the storage layer.

Currently, a full-link anomaly monitoring system is usually deployed in a storage-computing separation architecture to discover and automatically diagnose abnormal IO requests, and to issue alarms based on abnormal IO requests.

However, as the magnitude of IO requests continues to rise, the number of alarms also continues to increase. The massive amount of alarm information leads to the accumulation of alarms, which puts a lot of alarm processing pressure and cannot handle alarms in a timely manner.

Contents of the invention

Various aspects of this application provide an alarm method, device and storage medium to improve alarm processing efficiency.

The embodiment of this application provides an alarm method, including:

When an alarm triggering event occurs, determine the request path corresponding to each abnormal IO request. A single request path represents the connectivity relationship between the physical nodes passed by the corresponding abnormal IO request;

Cluster abnormal IO requests based on the request path to generate at least one abnormal IO request group, where the abnormal IO requests corresponding to the request paths passing through the same physical node are located in the same abnormal IO request group;

Output alarm information based on abnormal IO request groups.

Further, determine the request paths corresponding to each of the abnormal IO requests, including:

Obtain the abnormal monitoring information corresponding to each of the abnormal IO requests;

From the abnormal monitoring information, the identification information and the order of the physical nodes passed by the corresponding abnormal IO requests are analyzed to determine the request paths corresponding to each of the abnormal IO requests.

Further, obtain the abnormal monitoring information corresponding to each of the abnormal IO requests, including:

Send an exception monitoring information acquisition request to the anomaly monitoring system, where the acquisition request carries the target anomaly type, wherein the anomaly monitoring system has diagnosed the anomaly type corresponding to each of the abnormal IO requests;

Receive exception monitoring information returned by the anomaly monitoring system corresponding to an abnormal IO request diagnosed as the target anomaly type.

Further, the abnormal monitoring information uses tracking trace information. The trace information corresponds to the IO request one-to-one. The trace information contains multiple span items with a sequential relationship. The span items correspond to the physical nodes through which the IO request passes. The span items The item contains the identification information of the corresponding physical node, and the order relationship between the span items contained in the trace information is used to represent the order of the physical nodes passed by the IO request.

Further, abnormal IO requests are clustered based on the request path to generate at least one abnormal IO request group, including:

Cluster the abnormal IO requests corresponding to each request path that can be connected through the physical node into abnormal IO request groups, or,

Query the physical nodes that can connect multiple request paths as clustering nodes; cluster the abnormal IO requests corresponding to each request path that can be connected by a single clustering node into abnormal IO request groups.

Furthermore, abnormal IO requests corresponding to various request paths that can be connected through physical nodes are clustered into abnormal IO request groups, including:

Using the physical nodes on each request path as vertices and the connectivity relationships between physical nodes on each request path as edges, an undirected graph is constructed;

From the undirected graph, search for connected components;

The abnormal IO requests corresponding to the request paths included in a single connected component are clustered into abnormal IO request groups.

Further, from the undirected graph, search for connected components, including:

When traversing to a target vertex in the undirected graph, search for the connected component where the target vertex is located;

Delete the connected component where the target vertex is located from the undirected graph;

From the remaining vertices in the undirected graph, continue to determine the next target vertex and search and delete the corresponding connected components until there are no remaining vertices in the undirected graph;

Output the searched connected components.

Furthermore, alarm information is output based on the abnormal IO request group, including:

After deduplicating the physical nodes passed by each abnormal IO request in the target abnormal IO request group, determine the remaining physical nodes as the target nodes;

Based on the identification information of the target node, the identification information of the cluster to which the target node belongs, and/or the exception type involved in the target abnormal IO request group, output alarm information for the target abnormal IO request group;

Wherein, the target abnormal IO request group is any abnormal IO request group.

Further, after deduplicating the physical nodes passed by each abnormal IO request in the target abnormal IO request group, the remaining physical nodes are determined as target nodes, including:

If an undirected graph is constructed based on each request path and connected components are searched from the undirected graph to cluster the target abnormal IO request group, then the connected components contained in the target abnormal IO request group are The physical node represented by each vertex serves as the target node.

Further, the physical nodes include at least computing nodes and storage nodes. After outputting the alarm information, the method further includes:

In response to the alarm processing instructions, analyze the connectivity structure formed between the computing node and the storage node under the target abnormal IO request group corresponding to the target alarm information;

Based on the pointing relationship between the connected structure and the abnormal node, the abnormal node causing the abnormality is inferred under the target abnormal IO request group.

Furthermore, based on the pointing relationship between the connection structure and the abnormal node, the abnormal node causing the abnormality is inferred under the target abnormal IO request group, including:

If there is a first type of connectivity structure under the target abnormal IO request group, the computing nodes in the first type of connectivity structure are inferred to be abnormal nodes. The first type of connectivity structure is a computing node that connects multiple storage nodes. ;or,

If there is a second type of connectivity structure under the target abnormal IO request group, the storage nodes in the second type of connectivity structure are inferred to be abnormal nodes. The second type of connectivity structure is a storage node connecting multiple computing nodes. ;or,

If there is a third type of connectivity structure under the target abnormal IO request group, the intermediate nodes in the third type of connectivity structure are inferred to be abnormal nodes. The third type of connectivity structure is multiple computing nodes and multiple storage nodes. Nodes are connected through intermediate nodes.

Further, the target exception type includes an IO unavailable class or an IO damaged class, and the physical nodes include computing nodes in the computing system, storage nodes in the storage system, and/or intermediate nodes used for network connection. The abnormality An IO request is an IO request initiated by a computing node in the computing system to a storage node in the storage system and in which an exception has occurred.

An embodiment of the present application also provides an electronic device, including a memory and a processor;

The memory is used to store one or more computer instructions;

The processor is coupled to the memory and is used to execute the one or more computer instructions to execute the aforementioned alarm method.

Embodiments of the present application also provide a computer-readable storage medium that stores computer instructions. When the computer instructions are executed by one or more processors, they cause the one or more processors to execute the foregoing data processing method.

An embodiment of the present application also provides a computer program product, including a computer program/instruction, wherein when the computer program is executed by a processor, it causes the processor to implement the foregoing alarm method.

In the embodiment of this application, it is proposed that when an alarm triggering event occurs, request paths are determined for abnormal IO requests, and abnormal IO requests are clustered based on the request paths to generate at least one abnormal IO request group passing through the same physical The abnormal IO requests corresponding to the node's request path are located in the same abnormal IO request group; moreover, it is proposed to output alarm information in units of abnormal IO request groups. In this way, from the perspective of a single abnormal physical node, the request paths of abnormal IO requests caused by this physical node all pass through this physical node, which allows the abnormal IO requests corresponding to these request paths to be clustered in the same Within the abnormal IO request group, therefore, it can be ensured that abnormal IO requests caused by the same abnormal cause can be alarmed in one alarm message. This can avoid repeated alarms for the same abnormal cause, thus reducing the number of alarm messages and thus making the alarm messages It can be processed in a timely manner and improve the efficiency of alarm processing.

Description of drawings

The drawings described here are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the attached picture:

Figure 1 is a schematic flow chart of an alarm method provided by an exemplary embodiment of the present application;

Figure 2 is a schematic diagram of an exemplary request path of an abnormal IO request provided by an exemplary embodiment of the present application;

Figure 3 is a logical schematic diagram of an exemplary clustering scheme provided by an exemplary embodiment of the present application;

Figure 4 is a schematic flow chart of another alarm method provided by an exemplary embodiment of the present application;

Figure 5 is a schematic flow chart of yet another alarm method provided by an exemplary embodiment of the present application;

Figure 6 is a schematic diagram of several exemplary connection structures provided by an exemplary embodiment of the present application;

FIG. 7 is a schematic structural diagram of an electronic device provided by another exemplary embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

As mentioned in the background art, in current alarm solutions for IO requests, alarm information is usually output in units of IO requests. Therefore, as the number of IO requests continues to increase, the resulting alarm information also increases massively. The continuous accumulation of alarm information puts great pressure on alarm processing, and alarms cannot be processed in a timely manner, resulting in inefficient alarm processing.

To this end, embodiments of this application propose a new alarm method. The basic idea is to cluster abnormal IO requests to generate abnormal IO request groups, and output alarm information in units of abnormal IO request groups. In this way, the amount of alarm information can be effectively reduced, ensuring timely access to alarm information, thereby improving alarm processing efficiency.

The technical solutions provided by each embodiment of the present application will be described in detail below with reference to the accompanying drawings.

Figure 1 is a schematic flow chart of an alarm method provided by an exemplary embodiment of the present application. The method can be executed by an alarm device. The alarm device can be implemented as software, hardware, or a combination of software and hardware. The alarm device can be integrated into an electronic device. in the device. Referring to Figure 1, the method may include:

Step 100: When an alarm triggering event occurs, determine the request path corresponding to each abnormal IO request. A single request path represents the connectivity relationship between the physical nodes passed by the corresponding abnormal IO request;

Step 101: Cluster abnormal IO requests based on the request path to generate at least one abnormal IO request group, where the abnormal IO requests corresponding to the request paths passing through the same physical node are located in the same abnormal IO request group;

Step 102: Output alarm information based on abnormal IO request groups.

The alarm method provided in this embodiment can be applied to various scenarios that require abnormal alarms for IO requests. In different scenarios, the initiator and receiver corresponding to the IO request may be different. For example, in the cloud storage scenario mentioned in the background art, the initiator of the IO request is usually the computing node in the computing system, and the recipient is usually the storage node in the storage system. For IO requests in other scenarios, no more examples of initiators and receivers are provided here. It should be understood that this embodiment does not limit application scenarios. In application scenarios where IO requests pass through multiple physical nodes, the alarm method provided by this embodiment can be used to improve alarm processing efficiency.

Referring to Figure 1, in step 100, the alarm triggering event may be receiving an alarm triggering instruction, or reaching a preset alarm period, or of course other types of events. This embodiment does not specify the event type of the alarm triggering event. There are no restrictions, and accordingly, there are no restrictions on the timing of starting the alarm method.

The abnormal IO request in step 100 refers to an IO request that has been diagnosed as abnormal. Regarding the abnormal diagnosis process of IO requests, this embodiment does not limit it. In this embodiment, it is supported to use existing or future IO request exception monitoring means to perform abnormal monitoring on IO requests, so as to detect abnormal IO requests in a timely manner. For example, in cloud storage scenarios, an anomaly monitoring system based on full-link tracking has been deployed. This anomaly monitoring system can generate trace information for IO requests and automatically diagnose the abnormal types corresponding to IO requests. Of course, this is only exemplary. In this embodiment, other abnormality monitoring means are also supported to discover abnormal IO requests in advance.

In this embodiment, only abnormal IO requests have alarm requirements. Therefore, in step 100, abnormal IO requests can be screened out and used as processing objects in this embodiment.

On this basis, in step 100, the request paths corresponding to each abnormal IO request can be determined. Among them, the request path is used to represent the connectivity relationship between the physical nodes passed by the corresponding abnormal IO request. It should be understood that the request path at least includes the physical node that initiates the abnormal IO request and the physical node that responds to the abnormal IO request. Of course, the request path may also include a physical node used for intermediate forwarding of the abnormal IO request.

Figure 2 is a schematic diagram of an exemplary request path of an abnormal IO request provided by an exemplary embodiment of the present application. Referring to Figure 2, the storage and computing separation architecture can include a computing system and a storage system. The computing system can include computing nodes, and the storage system can include storage nodes. For different cloud storage products, different node organizational structures may be used in the provided storage systems. For example, as shown in Figure 2, for block storage products, at least two types of storage nodes can be deployed in the storage system provided: the first type of storage node is installed with a data block server (blockserver) for scheduling storage resources; the second type A chunkserver for data storage is installed on the storage node. That is to say, the data related to the IO request is stored on the second-class storage node in the storage system, and the first-class storage node is mainly responsible for the third-class storage node. The storage resources on the second-category storage nodes are scheduled and managed and are not responsible for data storage. Of course, the storage and computing separation architecture shown in Figure 2 is only exemplary, and the storage system may not include the first type of storage nodes, which is not limited here.

Continuing to refer to Figure 2, the abnormal IO request shown in Figure 2 is issued by the computing node A in the computing system, is forwarded by the storage node 1 in the storage system, and finally reaches the storage node 3 in the storage system and is sent by the storage node 3. Complete response. Based on this, the request path corresponding to the abnormal IO request can be determined as: computing node A-storage node 1-storage node 2. It can be understood that the request path not only represents the physical node that the abnormal IO request passes through, but also represents the connectivity relationship between the physical nodes that it passes through. Referring to Figure 2, the request path corresponding to the abnormal IO request can indicate that computing node A is connected to storage node 1, and storage node 1 is connected to storage node 2.

During the research process, the inventor found that the request paths corresponding to different abnormal IO requests may pass through the same physical nodes, and the request paths can be connected based on such physical nodes.

On this basis, this embodiment proposes in step 101 that abnormal IO requests can be clustered based on the request path to generate at least one abnormal IO request group. Among them, the abnormal IO requests corresponding to the request paths passing through the same physical node are located in the same abnormal IO request group. It should be understood that in step 102, the abnormal IO requests are clustered, so that at least one abnormal IO request group can be generated. The basis for clustering is the connectivity relationship between request paths.

As mentioned earlier, request paths can be connected based on the same physical node they pass through. Therefore, each request path connected based on the same physical node can be clustered together, which can ensure that exceptions passing through the same physical node IO requests can be clustered within the same abnormal IO request group. In this way, based on the connectivity relationship between the request paths, each request path determined in step 100 will be classified under each clustered abnormal IO request group.

During the research process, the inventor found that the first request path may be connected to multiple other request paths, and the physical nodes on which the connections between the multiple other request paths and the first request path are based may be different. In a preferred implementation manner, in step 101, abnormal IO requests corresponding to each request path that can be connected based on physical nodes are clustered into abnormal IO request groups. In this preferred implementation manner, multiple other request paths that are connected to the first request path based on different physical nodes can be clustered under the same abnormal IO request group. The first request path may be any request path determined in step 100.

For example, the first request path may be: A-B, based on physical node A, and other request paths connected to the first request path include: A-C and A-D-F; and based on physical node B, other request paths connected to the first request path There are: B-G. Therefore, although request paths B-G and request paths A-C do not pass through the same physical node, they are both connected to the first request path and can be clustered into the same abnormal IO request group.

In this preferred implementation, multiple clustering schemes can be used to implement clustering of abnormal IO requests. Figure 3 is a logical schematic diagram of an exemplary clustering scheme provided by an exemplary embodiment of the present application. Referring to Figure 3, in this exemplary clustering scheme: the physical nodes on each request path can be used as vertices, and the connectivity relationships between physical nodes on each request path can be used as edges to construct an undirected graph; from the undirected graph , search for connected components; cluster the abnormal IO requests corresponding to the request paths contained in a single connected component into abnormal IO request groups.

In this exemplary clustering scheme, a data structure such as an undirected graph is introduced to carry each request path. Among them, Graph is a non-linear structure, including two elements: vertices and edges. Edges are used to represent the connection relationships between vertices. For a graph, if the edges it contains have no direction, it can be called an undirected graph. In this exemplary clustering scheme, the connectivity relationship between the physical nodes in the request path can be represented through an undirected graph, and whether the request paths can be connected can also be represented. Based on this, it is proposed in this exemplary clustering scheme that connected components can be searched from the constructed undirected graph. A connected component refers to a connected maximal subgraph in an undirected graph, that is, there is a reachable path between any two vertices in a connected component. Referring to Figure 3, an undirected graph constructed in this exemplary clustering scheme is shown. It can be understood that in this undirected graph, not all reachable paths exist between any two vertices, for example, There is no reachable path between 6 and vertex 7. Referring to Figure 3, three connected components searched from the undirected graph are also shown.

In this exemplary clustering scheme, the implementation logic for searching connected components can be:

When traversing to the target vertex in the undirected graph, search for the connected component where the target vertex is located;

Remove the connected component where the target vertex is located from the undirected graph;

From the remaining vertices in the undirected graph, continue to determine the next target vertex and search and delete the corresponding connected components until there are no remaining vertices in the undirected graph;

Output the searched connected components.

In practical applications, the IP address of the physical node can be used as identification information. In this way, in the undirected graph, each vertex can be marked as an IP address. Based on this, each IP address in the undirected graph can be traversed, and then traversed to the target When using an IP address, the connected component of the target vertex corresponding to the target IP address can be searched. Here, the search algorithm used in the process of searching for the connected component of the target vertex is not limited. For example, the depth first search (DFS) algorithm can be used, and the search principle will not be described in detail here. Afterwards, the searched connected components can be deleted from the undirected graph, and the next target IP address can be determined among the remaining IP addresses. In this cycle, all connected components in the undirected graph can be searched.

In this way, the connectivity between each request path can be accurately characterized based on the undirected graph, and the clustering problem between request paths can be converted into a problem of searching connected components in the undirected graph, thereby effectively improving the request path The clustering efficiency between request paths is essentially the clustering between abnormal IO requests. Therefore, by introducing an undirected graph, the clustering efficiency of abnormal IO requests can be effectively improved and the same path can be ensured. Abnormal IO requests of physical nodes can be clustered in the same abnormal IO request group.

It is worth noting that the clustering scheme provided in Figure 3 is only exemplary. In this embodiment, other clustering schemes can also be used to implement the clustering of abnormal IO requests in step 101. For example, you can traverse each request path, and when traversing to the target request path, search for other request paths that have the same physical node as the target request path, and record the connectivity relationship between the target request path and these other request paths, and then continue Traverse the next request path, and loop like this to obtain the request paths connected to each request path. On this basis, you can select a request path as the starting path, add the starting path and its connected request paths to the path group, and then continue to use each request path connected by this request path as the starting path. Starting path, continue to add the request paths connected by each starting path to the path group. The request paths already in the path group do not need to be added repeatedly. In this cycle, the connected request paths can be clustered into the same path. group. This clustering scheme can obtain the same clustering effect as in Figure 3. No further examples of clustering schemes are provided here, and this embodiment is not limited thereto.

In the above preferred implementation, abnormal IO requests corresponding to each directly or indirectly connected request path can be clustered into the same abnormal IO request group. This not only ensures that abnormal IO requests passing through the same physical node can be clustered. The classes are in the same abnormal IO request group, and it is guaranteed that each abnormal IO request only appears in one abnormal IO request group. Therefore, repeated analysis of the same abnormal IO request can be avoided, thereby better reducing clustered exceptions. The number of IO request groups.

In addition to the preferred implementation methods provided above, in this embodiment, other implementation methods may also be used to cluster abnormal IO request groups in step 101.

For example, in another optional implementation: query physical nodes that can connect multiple request paths as cluster nodes; cluster abnormal IO requests corresponding to each request path that a single cluster node can connect For abnormal IO request group. This implementation method also ensures that abnormal IO requests passing through the same physical node can be clustered in the same abnormal IO request group, which allows subsequent output alarm information to point to a single exception cause, making alarm processing easier. However, the number of clustered abnormal IO request groups will be more than the aforementioned preferred implementation method. In addition, in this optional implementation, the aforementioned undirected graph method can also be used to cluster heterogeneous IO request groups. For example: when traversing to the target vertex in the undirected graph, if there are multiple If the request path passes through the target vertex, the abnormal IO requests corresponding to the multiple request paths passing through the target vertex are clustered into abnormal IO request groups; the multiple request paths passing through the target vertex are deleted from the undirected graph; from the undirected graph Among the remaining vertices, continue to determine the next target vertex and search and delete multiple request paths passing through the target vertex until there are no remaining vertices in the undirected graph to obtain at least one abnormal IO request group.

It can be understood that no matter which implementation method is adopted, in this embodiment, at least one abnormal IO request group can be clustered in step 101, and, from the perspective of a single physical node, exceptions passing through the same physical node IO requests can be clustered within the same abnormal IO request group. During the research process, the inventor found that when a physical node is abnormal, the IO requests that need to pass through the physical node will most likely be abnormal. In step 101 of this embodiment, the abnormal IO requests that pass through this type of physical node are already detected. Clustering into the same abnormal IO request group can essentially be understood as clustering abnormal IO requests caused by the same exception into the same abnormal IO request group.

On this basis, referring to Figure 1, this embodiment proposes in step 102 to output alarm information in units of abnormal IO request groups. That is, an abnormal IO request group can output an alarm message. In this embodiment, the alarm content contained in the alarm information is not limited, as long as it can provide the necessary content required for alarm processing under the abnormal IO request group. In this embodiment, the content fields in the alarm information can be configured on demand. In step 102, the corresponding alarm content can be generated based on the required content fields in the alarm information, and the generated alarm content can be encapsulated in the corresponding alarm information. Content field to generate and output alarm information. The construction scheme of the alarm information will be exemplified later and will not be described in detail here.

Following the above mentioned, abnormal IO requests caused by the same abnormal reason are clustered in the same abnormal IO request group. Here, in step 102, abnormal IO requests caused by the same abnormal reason can be alarmed in one alarm message. , which can avoid repeated alarms for the same abnormality cause. During the research process, the inventor found that the number of abnormal IO request groups clustered based on step 101 in this embodiment is much smaller than the number of abnormal IO requests. Therefore, the number of alarm information output in step 102 will be much smaller than the number of abnormal IO request groups. The traditional number of alarm messages output in units of IO requests.

In summary, in this embodiment, it is proposed that when an alarm triggering event occurs, request paths are determined for abnormal IO requests, and abnormal IO requests are clustered based on the request paths to generate at least one abnormal IO request group passing through the same The abnormal IO requests corresponding to the request paths of the physical nodes are located in the same abnormal IO request group; moreover, it is proposed to output alarm information in units of abnormal IO request groups. In this way, from the perspective of a single abnormal physical node, the request paths of abnormal IO requests caused by this physical node all pass through this physical node, which allows the abnormal IO requests corresponding to these request paths to be clustered in the same Within the abnormal IO request group, therefore, it can be ensured that abnormal IO requests caused by the same abnormal cause can be alarmed in one alarm message. This can avoid repeated alarms for the same abnormal cause, thus reducing the number of alarm messages and thus making the alarm messages It can be processed in a timely manner and improve the efficiency of alarm processing.

Figure 4 is a schematic flowchart of another alarm method provided by an exemplary embodiment of the present application. Referring to Figure 4, the method may include:

Step 400: When an alarm triggering event occurs, obtain the abnormal monitoring information corresponding to each abnormal IO request;

Step 401: From the abnormal monitoring information, parse the identification information and the order of the physical nodes passed by the corresponding abnormal IO requests to determine the corresponding request paths of the abnormal IO requests;

Step 402: Cluster the abnormal IO requests based on the request paths to generate at least one abnormal IO request group, where the abnormal IO requests corresponding to the request paths passing through the same physical node are located in the same abnormal IO request group;

Step 403: Output alarm information based on abnormal IO request groups.

For steps 402 and 403, reference may be made to the relevant descriptions in the foregoing embodiments and will not be repeated here. In this embodiment, based on step 400 and step 401, an optional implementation method of determining the request path corresponding to the abnormal IO request is provided. This optional implementation solution can be combined with the implementation solutions provided for other steps in the above or following embodiments to generate new technical solutions.

Referring to Figure 4, in this optional implementation, the abnormal monitoring information corresponding to each abnormal IO request can be obtained. As mentioned before, this embodiment supports various abnormality monitoring methods for IO requests, and all of these abnormality monitoring methods can produce abnormality monitoring information. Preferably, in this embodiment, an abnormality monitoring method capable of producing abnormality monitoring information in units of IO requests may be used. Of course, this is only preferred. In the case where the abnormal monitoring data is generated in other units, this embodiment supports organizing these abnormal monitoring data into abnormal monitoring information in IO request units.

In an exemplary solution: the exception monitoring information uses tracking trace information. The trace information corresponds to the IO request one-to-one. The trace information contains multiple span items that have a sequential relationship. The span items correspond to the physical nodes that the IO requests pass through. One-to-one correspondence, the span item contains the identification information of the corresponding physical node, and the sequence relationship between the span items contained in the trace information is used to represent the path order between the physical nodes that the IO request passes through. In this exemplary solution, in step 400, the trace information corresponding to the abnormal IO requests can be obtained from the anomaly monitoring system based on full-link tracking. As mentioned above, this anomaly monitoring system can obtain IO requests. Generate trace information for the unit and automatically diagnose the exception type corresponding to the IO request. Moreover, this anomaly monitoring system constructs span items for each physical node that the IO request passes through, which is used to describe the IO processing process on the physical node. There is a sequential relationship between the span items included in the trace information, which can be used to determine the IO The basis for the order in which the request passes between the physical nodes. It should be understood that this is only exemplary. In this embodiment, other types of abnormality monitoring information can also be used, but are not limited thereto.

On this basis, with reference to Figure 4, it is proposed in step 401 that the identification information and passing sequence of the physical nodes passed by the corresponding abnormal IO requests can be parsed from the abnormal monitoring information to determine the requests corresponding to each of the abnormal IO requests. path. During the research process, the inventor found that the abnormal monitoring information usually records the IO processing description information on each physical node through which the abnormal IO request passes, and the IO processing description information usually includes the identification information of the physical node. The physical node The identification information of the cluster to which it belongs, the IO processing time on the physical node, the identification information of the previous hop node of the physical node, the identification information of the next hop node of the physical node, the type of IO processing operation performed on the physical node, etc. information. For example, the span item mentioned above records the IO processing description information on the physical node.

Based on this, in step 401, the identification information and routing sequence of the physical nodes passed by the corresponding abnormal IO request can be analyzed from the abnormal monitoring information, and further, the request path corresponding to the abnormal IO request can be determined.

Furthermore, the inventor discovered during the research process that the exception types corresponding to different abnormal IO requests may not be exactly the same. In actual applications, the exception types of IO requests may include but are not limited to IO unavailable classes or IO damaged classes. Among them, the IO unavailable category usually means that the IO request has not completed the response, and the IO damaged category usually means that the IO request has completed the response but the response rate is too slow. It should be understood that the several exception types provided here are only exemplary, and this embodiment is not limited thereto. Moreover, the inventor also found that the exception type is an important reference in subsequent alarm processing links.

To this end, in this optional implementation, an exemplary acquisition scheme is proposed for the process of obtaining the abnormal monitoring information corresponding to each abnormal IO request: the abnormal monitoring information can be sent to the abnormal monitoring system deployed for the storage system Obtain the request, which carries the target exception type. The exception monitoring system has diagnosed the exception type corresponding to the abnormal IO request; receives the exception corresponding to the abnormal IO request returned by the exception monitoring system and has been diagnosed as the target exception type. Monitoring information.

In this exemplary acquisition scheme, the target anomaly type is carried in the anomaly monitoring information acquisition request to the anomaly monitoring system. Since the anomaly type corresponding to the abnormal IO request has been diagnosed in the anomaly monitoring system, the anomaly monitoring system can Filter out abnormal IO requests that have been diagnosed as the target exception type and return the exception monitoring information corresponding to the filtered abnormal IO requests.

In this way, based on this exemplary acquisition scheme, in the alarm scheme provided by this embodiment, first, a layer of clustering is performed on abnormal IO requests from the dimension of abnormal type, and abnormal IO requests diagnosed as the same abnormal type are clustered in Together; secondly, according to the concept of clustering based on request paths provided in step 402, abnormal IO requests under different exception types can be re-clustered respectively. In this way, abnormal IO request groups can be clustered under different exception types. That is, the exception types corresponding to each abnormal IO request in the same abnormal IO request group will be consistent. Furthermore, the single output in step 403 The alarm information will only involve one exception type. This can provide a reference for exception types for subsequent alarm processing.

Of course, in this optional implementation solution, other exemplary solutions can also be adopted to support displaying the exception type in the alarm information. For example, the abnormal monitoring data corresponding to all abnormal IO requests can be obtained from the abnormal monitoring system, and processed uniformly according to steps 401 to 403. However, in step 403, a single abnormal IO request group can be re-clustered according to the exception type, and the abnormal IO requests involved in different exception types can be recorded in the alarm information. This can also provide a reference for the exception type for subsequent alarm processing links. No more examples will be given here to support the display of exception types in alarm information, and this embodiment is not limited to this.

In summary, in this embodiment, the abnormal monitoring information corresponding to each abnormal IO request can be obtained, and the request path corresponding to each abnormal IO request can be accurately determined based on the abnormal monitoring information, which can then provide a method for clustering abnormal IO requests. Accurate basis. Moreover, it is also proposed that abnormal IO requests can be clustered from the dimension of exception type, thereby supporting the reasonable display of exception types in alarm information, and providing a reference basis for subsequent alarm processing links, which can further improve the understanding of alarms. processing efficiency.

In the above or following embodiments, multiple implementation methods may be used to realize the construction of alarm information. Since the logic of constructing alarm information under each abnormal IO request group is consistent, for the convenience of description, the following takes the target abnormal IO request group as an example to describe the alarm information construction plan. It should be understood that the target abnormal IO request group can be any clustered abnormal IO request group.

In an optional implementation method: after deduplicating the physical nodes passed by each abnormal IO request in the target abnormal IO request group, the remaining physical nodes can be determined as the target nodes; based on the identification information of the target node, the target The identification information of the cluster to which the node belongs and/or the exception type involved in the target abnormal IO request group is the target abnormal IO request group, and alarm information is output.

As mentioned above, different request paths may pass through the same physical node. For this reason, deduplication of physical nodes is proposed in this optional implementation. Through deduplication processing, the physical nodes involved in the target abnormal IO request group can be determined. In an exemplary deduplication solution: if an undirected graph is constructed based on each request path and connected components are searched from the undirected graph to cluster the target abnormal IO request group, then the connected components corresponding to the target abnormal IO request group are The physical node represented by each vertex contained in the component is used as the target node.

It can be understood that, in this optional implementation, the alarm information may include at least a content field for recording the identification information of the target node, based on the identification information of the cluster to which the target node belongs, and/or for recording The content field of the exception type involved in the target exception IO request group. Of course, these content fields are only exemplary, and this embodiment is not limited thereto.

The aforementioned identification information of the target node and the identification information of the cluster to which the target node belongs can be obtained from the corresponding abnormal monitoring information of the abnormal IO requests included in the target abnormal IO request group. For example, it can be obtained from the span item mentioned above. obtained. As for the exception types involved in the target abnormal IO request, you can refer to the exemplary solution provided above. Before clustering the abnormal IO requests based on the request path, first perform a layer of clustering on the abnormal IO requests based on the exception type. This way In this case, each abnormal IO request clustered based on the request path under the target exception type can be marked as the target exception type, and then the exception type marked as the target abnormal IO request group can be carried in the alarm information. If the abnormal IO requests are clustered based on the request path, and then re-clustered based on the exception type within the target abnormal IO request group, the clustered exception types can be marked for the target abnormal IO request group and recorded separately. Abnormal IO requests under the exception type, and then the alarm information carries the exception type marked for the target abnormal IO request group and the abnormal IO requests under each exception type.

It should be understood that other implementation methods can be used to construct alarm information in this embodiment, and the content fields included in the alarm information are not limited to the several exemplary content fields provided above. Based on the alarm information construction solution provided in this embodiment, the alarm information can be used to prompt which clusters and/or which nodes have experienced which type of IO abnormality. It can be seen that the alarm information in this embodiment not only prompts exceptions in units of IO requests, but also prompts exceptions from the dimensions of exception type, cluster and node, etc. This can make it easier to locate exceptions in the subsequent alarm processing process, and thus can Further improve the efficiency of alarm processing.

Figure 5 is a schematic flowchart of yet another alarm method provided by an exemplary embodiment of the present application. Referring to Figure 5, the method may include:

Step 500: When an alarm triggering event occurs, determine the request paths corresponding to each of the abnormal IO requests. A single request path represents the connectivity relationship between the physical nodes passed by the corresponding abnormal IO requests;

Step 501: Cluster abnormal IO requests based on the request path to generate at least one abnormal IO request group, where the abnormal IO requests corresponding to the request paths passing through the same physical node are located in the same abnormal IO request group;

Step 502: Output alarm information based on abnormal IO request groups.

Step 503: In response to the alarm processing instruction, analyze the connection structure formed between the computing node and the storage node under the target abnormal IO request group corresponding to the target alarm information;

Step 504: Based on the pointing relationship between the connectivity structure and the abnormal node, infer the abnormal node that caused the abnormality under the target abnormal IO request group.

For steps 500 to 502, reference may be made to the relevant descriptions in the foregoing embodiments, which will not be repeated here. In this embodiment, based on step 503 and step 504, an optional implementation solution after outputting the alarm information is provided. This optional implementation solution can be combined with the implementation solutions provided for other steps in the above or following embodiments to generate new technical solutions.

Referring to Figure 5, after the alarm information is output, the alarm processing logic for each piece of output alarm information can be started in response to the alarm processing instruction. Since the alarm processing logic implemented for different alarm information is consistent, for convenience of description, in this embodiment, the alarm processing logic is explained by taking the target alarm information as an example. It should be understood that the target alarm information may be any alarm information output in step 502 in this embodiment.

Referring to Figure 5, in this embodiment, considering that the nature of IO requests can be understood as data read and write requests, there are at least physical nodes for data storage. In this embodiment, the nodes used for data storage are described as storage nodes. ; The root cause of data reading and writing is usually used for calculation. Therefore, in this embodiment, the physical node that initiates the IO request is described as a computing node. In this way, in this embodiment, the physical nodes in the request path may include at least computing nodes and storage nodes. As mentioned earlier, the request path can also include intermediate nodes for intermediate forwarding of IO requests, etc. In the storage-computing separation architecture, computing nodes can be located in the computing system, and storage nodes can be located in the storage system. Of course, in other scenarios, the deployment locations of computing nodes and storage nodes may not be limited to this, and the deployment locations of computing nodes and storage nodes are not limited here.

Based on this, it is proposed in step 503 that the connection structure formed between the computing node and the storage node can be analyzed under the target abnormal IO request group corresponding to the target alarm information. The connectivity structure in this embodiment can be understood as a structure formed based on request paths connected by a physical node. The physical node can be a computing node, a storage node, or it can also be used for IO request forwarding. the intermediate node. During the research process, the inventor found that under a single abnormal IO request group, multiple connected structures may be analyzed.

Figure 6 is a schematic diagram of several exemplary connection structures provided by an exemplary embodiment of the present application. Referring to Figure 6, in this embodiment, there can be at least three types of connected structures:

The first type of connectivity structure is a computing node connected to multiple storage nodes;

The second type of connectivity structure is a storage node connected to multiple computing nodes;

The third type of connectivity structure is that multiple computing nodes and multiple storage nodes are connected through intermediate nodes. The intermediate node may be a network device used for network transfer, etc.

To this end, it is proposed in step 504 that the pointing relationship between the connected structure and the abnormal node can be configured in advance. This pointing relationship can be used to guide the location of abnormal nodes under different connectivity structures. In this way, in step 504, based on the pointing relationship, the abnormal node causing the exception can be inferred under the target abnormal IO request group. If multiple connected structures are analyzed under the target abnormal IO request group, the abnormal nodes can be inferred based on the pointing relationships under the multiple connected structures.

In an exemplary speculative scenario:

If there is a first-type connected structure under the target abnormal IO request group, the computing nodes in the first-type connected structure are inferred to be abnormal nodes; or,

If there is a second type of connected structure under the target abnormal IO request group, the storage node in the second type of connected structure is inferred to be an abnormal node; or,

If there is a third type of connectivity structure under the target abnormal IO request group, the intermediate nodes in the third type of connectivity structure are inferred to be abnormal nodes.

Referring to Figure 6, for the first type of connectivity structure, if multiple storage nodes are connected to the same computing node, the cause of the abnormality can be initially located on the computing node. Usually, the abnormality of the computing node causes the connection to the computing node. The IO requests corresponding to multiple storage nodes are abnormal. Similarly, for the second type of connected structure, the cause of the abnormality can be initially located at the storage node in this type of connected structure. As for the third type of connection structure, since the IO requests between multiple computing nodes and multiple storage nodes are abnormal, the cause of the abnormality can be initially located in the network used to connect the multiple computing nodes and the multiple storage nodes. On the intermediate node of the node, it is usually due to an abnormality in the intermediate node that the multiple IO requests passing through the intermediate node are abnormal.

It is worth noting that in this embodiment, the alarm processing logic provided is only exemplary. Moreover, based on this exemplary alarm processing logic, abnormal nodes can be inferred, and the inferred abnormal nodes can be used as reference for operation and maintenance. , in actual applications, more inference dimensions can be introduced to further correct the inference results provided by this embodiment. Of course, manual analysis and other processing steps can also be added to ensure that the cause of the anomaly is accurately located. Here, other inference dimensions and Manual analysis logic, etc. are not limited.

In summary, in this embodiment, after outputting the alarm information, alarm processing logic is also proposed. In the alarm processing logic, the request path and abnormal IO request group data generated during the alarm process in this embodiment can be fully utilized. , as the analysis basis in the alarm processing logic. Based on this, the connectivity structure can be analyzed under each alarm information, and abnormal nodes can be inferred based on the connectivity structure to provide a reference for operation and maintenance, so that alarm processing can be completed faster and more accurately, which can further improve alarm processing. processing efficiency.

It should be noted that some of the processes described in the above embodiments and drawings include multiple operations that appear in a specific order, but it should be clearly understood that these operations may not be performed in the order in which they appear in this article. Or execute in parallel. The sequence numbers of operations, such as 101, 102, etc., are only used to distinguish different operations. The sequence numbers themselves do not represent any execution order. Additionally, these processes can include more or fewer operations, and the operations can be performed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different connected structures, etc., and do not represent the order, nor do they limit "first" and "second" to be different type.

FIG. 7 is a schematic structural diagram of an electronic device provided by another exemplary embodiment of the present application. As shown in FIG. 7 , the electronic device may include: a memory 70 and a processor 71 .

The processor 71 is coupled to the memory 70 and is used to execute the computer program in the memory 70 for:

When an alarm triggering event occurs, determine the request paths corresponding to the abnormal IO requests. A single request path represents the connectivity relationship between the physical nodes that the corresponding abnormal IO requests pass through;

Cluster abnormal IO requests based on the request path to generate at least one abnormal IO request group, where the abnormal IO requests corresponding to the request paths passing through the same physical node are located in the same abnormal IO request group;

Output alarm information based on abnormal IO request groups.

In an optional embodiment, when determining the request path corresponding to each of the abnormal IO requests, the processor 71 may be specifically configured to:

Obtain the abnormal monitoring information corresponding to each of the abnormal IO requests;

From the abnormal monitoring information, the identification information and the order of the physical nodes passed by the corresponding abnormal IO requests are analyzed to determine the request paths corresponding to each of the abnormal IO requests.

In an optional embodiment, when obtaining the abnormal monitoring information corresponding to each of the abnormal IO requests, the processor 71 may be specifically configured to:

Send an exception monitoring information acquisition request to the anomaly monitoring system used for IO anomaly monitoring, where the acquisition request carries the target anomaly type, wherein the anomaly monitoring system has diagnosed the anomaly type corresponding to each of the abnormal IO requests;

Receive exception monitoring information returned by the anomaly monitoring system corresponding to an abnormal IO request diagnosed as the target anomaly type.

In an optional embodiment, the abnormality monitoring information uses tracking trace information. The trace information corresponds to the IO request one-to-one. The trace information contains multiple span items that have a sequential relationship. The span items are related to the physical location of the IO request. There is a one-to-one correspondence between nodes. The span item contains the identification information of the corresponding physical node. The sequence relationship between the span items contained in the trace information is used to represent the order of the physical nodes that the IO request passes through.

In an optional embodiment, when the processor 71 clusters abnormal IO requests based on request paths to generate at least one abnormal IO request group, it may be specifically configured to:

Cluster the abnormal IO requests corresponding to each request path that can be connected through the physical node into abnormal IO request groups, or,

Query the physical nodes that can connect multiple request paths as clustering nodes; cluster the abnormal IO requests corresponding to each request path that can be connected by a single clustering node into abnormal IO request groups.

In an optional embodiment, when the processor 71 clusters the abnormal IO requests corresponding to each request path that can be connected through the physical node into abnormal IO request groups, it may be specifically used to:

Using the physical nodes on each request path as vertices and the connectivity relationships between physical nodes on each request path as edges, an undirected graph is constructed;

From the undirected graph, search for connected components;

The abnormal IO requests corresponding to the request paths included in a single connected component are clustered into abnormal IO request groups.

In an optional embodiment, when searching for connected components from the undirected graph, the processor 71 may be specifically configured to:

When traversing to a target vertex in the undirected graph, search for the connected component where the target vertex is located;

Delete the connected component where the target vertex is located from the undirected graph;

From the remaining vertices in the undirected graph, continue to determine the next target vertex and search and delete the corresponding connected components until there are no remaining vertices in the undirected graph;

Output the searched connected components.

In an optional embodiment, when outputting alarm information based on abnormal IO request groups as a unit, the processor 71 may be specifically used to:

After deduplicating the physical nodes passed by each abnormal IO request in the target abnormal IO request group, determine the remaining physical nodes as the target nodes;

Based on the identification information of the target node, the identification information of the cluster to which the target node belongs, and/or the exception type involved in the target abnormal IO request group, output alarm information for the target abnormal IO request group;

Wherein, the target abnormal IO request group is any abnormal IO request group.

In an optional embodiment, after the processor 71 deduplicates the physical nodes passed by each abnormal IO request in the target abnormal IO request group, and determines the remaining physical nodes as the target nodes, the processor 71 may be specifically configured to: :

If an undirected graph is constructed based on each request path and connected components are searched from the undirected graph to cluster the target abnormal IO request group, then the connected components contained in the target abnormal IO request group are The physical node represented by each vertex serves as the target node.

In an optional embodiment, the physical node includes at least a computing node and a storage node. After outputting the alarm information, the processor 71 may also be configured to:

In response to the alarm processing instructions, analyze the connectivity structure formed between the computing node and the storage node under the target abnormal IO request group corresponding to the target alarm information;

Based on the pointing relationship between the connected structure and the abnormal node, the abnormal node causing the abnormality is inferred under the target abnormal IO request group.

In an optional embodiment, when the processor 71 infers the abnormal node that causes the exception under the target abnormal IO request group based on the pointing relationship between the connection structure and the abnormal node, it may be specifically used to:

If there is a first type of connectivity structure under the target abnormal IO request group, the computing nodes in the first type of connectivity structure are inferred to be abnormal nodes. The first type of connectivity structure is a computing node that connects multiple storage nodes. ;or,

If there is a second type of connectivity structure under the target abnormal IO request group, the storage nodes in the second type of connectivity structure are inferred to be abnormal nodes. The second type of connectivity structure is a storage node connecting multiple computing nodes. ;or,

If there is a third type of connectivity structure under the target abnormal IO request group, the intermediate nodes in the third type of connectivity structure are inferred to be abnormal nodes. The third type of connectivity structure is multiple computing nodes and multiple storage nodes. Nodes are connected through intermediate nodes.

In an optional embodiment, the target exception type includes an IO unavailable class or an IO damaged class, and the physical nodes include computing nodes in the computing system, storage nodes in the storage system, and/or network connections. Intermediate node, the abnormal IO request is an IO request initiated by the computing node in the computing system to the storage node in the storage system and in which an abnormality has occurred.

Further, as shown in FIG. 7 , the electronic device also includes: a communication component 72 and a power supply component 73 and other components. Only some components are schematically shown in FIG. 7 , which does not mean that the electronic device only includes the components shown in FIG. 7 .

It is worth noting that for the technical details of the various embodiments of the electronic device mentioned above, reference can be made to the relevant descriptions in the foregoing method embodiments. To save space, they will not be described in detail here, but this should not cause a loss of the protection scope of the present application. .

Correspondingly, embodiments of the present application also provide a computer-readable storage medium storing a computer program. When the computer program is executed, the steps performed in the above method embodiments can be implemented.

Correspondingly, embodiments of the present application also provide a computer program product, which can implement each step performed in the above method embodiment when the computer program in the computer program product is executed.

The memory in Figure 7 above is used to store computer programs, and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phonebook data, messages, pictures, videos, etc. Memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable memory Read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The above-mentioned communication component in Figure 7 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G/LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

The power supply component in Figure 7 above provides power to various components of the device where the power supply component is located. A power component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power component resides.

Those skilled in the art will understand that embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in a process or processes in a flowchart and/or a block or blocks in a block diagram.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in a process or processes in the flowchart and/or in a block or blocks in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

It should also be noted that the terms "comprises," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements not only includes those elements, but also includes Other elements are not expressly listed or are inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or device that includes the stated element.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all It is information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or reject.

The above descriptions are only examples of the present application and are not intended to limit the present application. To those skilled in the art, various modifications and variations may be made to this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included in the protection scope of this application.

Claims (15)

1. An alert method, comprising:

under the condition of an alarm triggering event, determining request paths corresponding to the abnormal IO requests respectively, wherein a single request path characterizes a communication relationship between physical nodes through which the corresponding abnormal IO requests pass;

clustering the abnormal IO requests based on the request paths to generate at least one abnormal IO request group, wherein the abnormal IO requests corresponding to the request paths passing through the same physical node are located in the same abnormal IO request group;

And outputting alarm information by taking the abnormal IO request group as a unit.

2. The method of claim 1, wherein determining the request paths to which the abnormal IO requests each correspond comprises:

acquiring the abnormality monitoring information corresponding to each abnormal IO request;

and analyzing the identification information and the passing sequence of the physical nodes passed by the corresponding abnormal IO requests from the abnormal monitoring information to determine the request paths corresponding to the abnormal IO requests.

3. The method of claim 2, wherein obtaining the anomaly monitoring information corresponding to each of the anomaly IO requests comprises:

an abnormal monitoring information acquisition request is sent to an abnormal monitoring system for carrying out IO abnormal monitoring, wherein the acquisition request carries a target abnormal type, and the abnormal monitoring system has diagnosed the abnormal type corresponding to each abnormal IO request;

and receiving the abnormality monitoring information corresponding to the abnormality IO request which is returned by the abnormality monitoring system and is diagnosed as the target abnormality type.

4. A method according to claim 2 or 3, wherein the anomaly monitoring information adopts trace information, the trace information corresponds to the IO request one by one, the trace information comprises a plurality of span items with sequence relations, the span items correspond to physical nodes through which the IO request passes one by one, the span items comprise identification information of the corresponding physical nodes, and the sequence relations among the span items contained in the trace information are used for representing the passing sequence among the physical nodes through which the IO request passes.

5. The method of claim 1, wherein clustering the abnormal IO requests based on the request paths to generate at least one abnormal IO request group comprises:

the abnormal IO requests corresponding to the request paths which can be communicated through the physical nodes are clustered into an abnormal IO request group, or,

inquiring a physical node capable of communicating a plurality of request paths as a clustering node; and clustering the abnormal IO requests corresponding to each request path which can be communicated by the single clustering node into an abnormal IO request group.

6. The method of claim 5, wherein clustering the abnormal IO requests corresponding to each request path that can be communicated through the physical node into an abnormal IO request group comprises:

taking physical nodes on each request path as vertexes, and taking a communication relation among the physical nodes on each request path as edges to construct an undirected graph;

searching for connected components from the undirected graph;

and clustering the abnormal IO requests corresponding to the request paths contained in the single connected component into an abnormal IO request group.

7. The method of claim 6, wherein searching for connected components from the undirected graph comprises:

Searching a connected component of a target vertex when traversing to the target vertex in the undirected graph;

deleting a connected component of the target vertex from the undirected graph;

continuing to determine the next target vertex from the residual vertices in the undirected graph, and searching and deleting the corresponding connected components until the residual vertices do not exist in the undirected graph;

outputting the searched connected component.

8. The method of claim 1, wherein outputting the alert information in units of the abnormal IO request group comprises:

after the physical nodes passed by each abnormal IO request in the target abnormal IO request group are de-duplicated, the rest physical nodes are determined as target nodes;

based on the identification information of the target node, the identification information of the cluster to which the target node belongs and/or the abnormality type related to the target abnormal IO request group, outputting alarm information for the target abnormal IO request group;

wherein the target abnormal IO request group is any abnormal IO request group.

9. The method of claim 8, wherein after de-duplicating the physical nodes traversed by each abnormal IO request in the target abnormal IO request group, determining the remaining physical nodes as target nodes comprises:

And if an undirected graph is constructed based on each request path and connected components are searched from the undirected graph to cluster out the target abnormal IO request group, taking physical nodes represented by all vertexes contained in the connected components corresponding to the target abnormal IO request group as target nodes.

10. The method of claim 1, wherein the physical nodes comprise at least a computing node and a storage node, and wherein after outputting the alert information, the method further comprises:

responding to an alarm processing instruction, and analyzing a communication structure formed between a computing node and a storage node under a target abnormal IO request group corresponding to target alarm information;

based on the pointing relation between the communication structure and the abnormal nodes, the abnormal nodes causing the abnormality are speculated under the target abnormal IO request group.

11. The method of claim 10, wherein speculating an exception node that caused an exception under the target exception IO request group based on a directed relationship between a communication structure and the exception node comprises:

if a first type of communication structure exists under the target abnormal IO request group, computing nodes in the first type of communication structure are presumed to be abnormal nodes, and the first type of communication structure is that one computing node is communicated with a plurality of storage nodes; or,

If a second type of communication structure exists under the target abnormal IO request group, presuming the storage nodes in the second type of communication structure as abnormal nodes, wherein the second type of communication structure is a storage node communicated with a plurality of computing nodes; or,

if a third type of communication structure exists under the target abnormal IO request group, the intermediate nodes in the third type of communication structure are presumed to be abnormal nodes, and the third type of communication structure is formed by communicating a plurality of computing nodes and a plurality of storage nodes through the intermediate nodes.

12. A method according to claim 3, wherein the target exception type comprises an IO unavailable class and/or an IO damaged class, the physical node comprises a computing node in a computing system, a storage node in a storage system and/or an intermediate node for network connection, and the abnormal IO request is an IO request that the computing node in the computing system initiates to the storage node in the storage system and has an abnormality occurred.

13. An electronic device comprising a memory and a processor;

the memory is used for storing one or more computer instructions;

the processor is coupled to the memory for executing the one or more computer instructions for performing the alerting method of any one of claims 1-12.

14. A computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the alerting method of any one of claims 1-12.

15. A computer program product comprising computer programs/instructions which, when executed by a processor, cause the processor to implement the alerting method of any one of claims 1-12.