CN116841718A - Physical resource scheduling method and scheduler based on Kubernetes - Google Patents

Abstract

The invention provides a physical resource scheduling method and a scheduler based on Kubernetes, wherein scheduling information in a Kubernetes cluster is obtained, and the scheduling information comprises Pod information of Pod to be scheduled and load information of each node; then traversing each node, inputting the Pod information of the Pod to be scheduled and the load information of the current node into a scheduling evaluation model, and obtaining the evaluation information of the Pod to be scheduled, which is output by the scheduling evaluation model, when the Pod to be scheduled is scheduled to the current node; the Pod to be scheduled is preferably scheduled based on the evaluation information corresponding to each node. The method introduces the load information of the current node, avoids the abnormal condition of physical resources on the node caused by considering only hardware resources, ensures the safe operation of the Kubernetes cluster, and improves the stability of the Kubernetes cluster.

Description

Physical resource scheduling method and scheduler based on Kubernetes

Technical Field

The present invention relates to the technical field of cloud computing, and in particular to a physical resource scheduling method and a scheduler based on Kubernetes.

Background Art

Kubernetes is a distributed system engine used to manage containerized applications on multiple hosts in a cloud platform. It can implement functions such as physical resource scheduling, automated deployment and operation, service discovery, elastic scaling, and high availability.

Physical resource scheduling can be implemented through the Kubernetes scheduler, and its core is how to select appropriate nodes from the cluster to allocate to work units (Pods). Currently, when implementing physical resource scheduling, the Kubernetes scheduler usually needs to include three stages: node pre-selection, node optimization, and node selection.

The Node pre-selection phase mainly checks the hardware resources of the Node through the Requests configuration in the corresponding Pod configuration, which is also called static resource checking. Since the Requests configuration used in static resource checking is mainly used to predict and set resources such as CPU and memory when configuring objects such as Deployment and StatusfulSet, and since the situations when the program is running are different, this will cause the Requests configuration to differ from the actual situation. That is, in the actual Pod running process, it may appear that a Node has a large amount of physical resources available from the Requests configuration, but in fact the Pod running on the Node has already occupied a large amount of physical resources on the Node, and the remaining physical resources on the Node are already relatively tight.

Furthermore, when a large number of Pods need to be scheduled in Kubernetes, it is easy for such seemingly idle Nodes to be judged as the preferred result due to the pre-selection method based on Requests configuration, resulting in more Pods being scheduled to the Node, causing abnormal physical resources on the Node. In serious cases, the Node may even go offline abnormally, which has an adverse impact on the overall operation security of Kubernetes and reduces the stability of the cluster.

Summary of the invention

The present invention provides a physical resource scheduling method and a scheduler based on Kubernetes, so as to solve the defects existing in the prior art.

The present invention provides a physical resource scheduling method based on Kubernetes, comprising:

Obtain scheduling information in the Kubernetes cluster, including Pod information of the Pod to be scheduled and load information of each node;

Traversing each of the nodes, inputting the Pod information of the Pod to be scheduled and the load information of the current node into the scheduling evaluation model, and obtaining the evaluation information of the Pod to be scheduled when being scheduled to the current node output by the scheduling evaluation model;

Scheduling the Pod to be scheduled based on the evaluation information corresponding to each of the nodes;

The scheduling evaluation model is trained based on a training sample set and evaluation information labels corresponding to each training sample in the training sample set. The training samples include Pod information of Pod samples and load information of node samples to which the Pod samples are scheduled.

According to a Kubernetes-based physical resource scheduling method provided by the present invention, the evaluation information label corresponding to each training sample is determined based on the following method:

Based on the density clustering algorithm, cluster analysis is performed on each of the training samples to obtain multiple clusters;

Based on the sample features in the multiple clusters, the training samples included in the multiple clusters are collectively labeled to obtain evaluation information labels corresponding to the training samples.

According to a Kubernetes-based physical resource scheduling method provided by the present invention, the density-based clustering algorithm performs cluster analysis on each of the training samples to obtain multiple cluster clusters, including:

Traversing each of the training samples, and judging whether the current training sample is a core sample based on the number of other training samples existing in a preset neighborhood of the current training sample; if the current training sample is a core sample, storing the current training sample in a core sample set;

Traversing each core sample in the core sample set, and determining whether other training samples existing in a preset neighborhood of the current core sample and the current core sample belong to the same initial cluster based on the degree of membership between the current core sample and other training samples existing in a preset neighborhood of the current core sample;

After determining the initial clusters corresponding to the nuclear samples in the nuclear sample set, calculating the silhouette coefficients between the initial clusters corresponding to the nuclear samples;

The preset parameters in the density clustering algorithm are adjusted, the traversal action is repeatedly performed, a plurality of silhouette coefficients are calculated, and the plurality of cluster clusters are determined based on the plurality of silhouette coefficients.

According to a Kubernetes-based physical resource scheduling method provided by the present invention, other training samples existing in a preset neighborhood of a current training sample are determined based on the following method:

Calculating the Hausdorff distance between the current training sample and each of the training samples;

If the Hausdorff distance is smaller than the preset radius of the preset neighborhood, it is determined that the training sample corresponding to the Hausdorff distance is other training samples existing in the preset neighborhood of the current training sample.

According to a Kubernetes-based physical resource scheduling method provided by the present invention, the degree of membership between other training samples existing in a preset neighborhood of the current core sample and the current core sample is determined based on the following method:

Traversing each other training sample existing in a preset neighborhood of the current core sample, and determining a number of adjacent samples that are adjacent to the other current training samples and are within the preset neighborhood of the current core sample;

The degree of membership between the current other training sample and the current core sample is determined based on the average, maximum and minimum values of the distances between the current other training sample and the plurality of adjacent samples.

According to a physical resource scheduling method based on Kubernetes provided by the present invention, the scheduling evaluation model is trained based on the following method:

Based on the training samples and the evaluation information labels corresponding to each training sample in the training sample set, a loss function including a regularization term is used to iteratively train multiple classification decision tree models to obtain the scheduling evaluation model.

According to a Kubernetes-based physical resource scheduling method provided by the present invention, during the iterative training process, the loss function used in the current round of iterative training is determined based on the first-order gradient and second-order gradient of the loss function used in the previous round of iterative training of the current round.

The present invention also provides a scheduler, comprising:

An information acquisition module is used to obtain scheduling information in the Kubernetes cluster, where the scheduling information includes Pod information of the Pod to be scheduled and load information of each node;

An evaluation module, used to traverse each of the nodes, input the Pod information of the Pod to be scheduled and the load information of the current node into a scheduling evaluation model, and obtain the evaluation information of the Pod to be scheduled when it is scheduled to the current node output by the scheduling evaluation model;

A scheduling module, used to schedule the Pod to be scheduled based on the evaluation information corresponding to each of the nodes;

The scheduling evaluation model is trained based on a training sample set and evaluation information labels corresponding to each training sample in the training sample set. The training samples include Pod information of Pod samples and load information of node samples to which the Pod samples are scheduled.

The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a Kubernetes-based physical resource scheduling method as described above is implemented.

The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described Kubernetes-based physical resource scheduling methods.

The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-described physical resource scheduling methods based on Kubernetes.

The physical resource scheduling method and scheduler based on Kubernetes provided by the present invention obtain the scheduling information in the Kubernetes cluster, and the scheduling information includes the Pod information of the Pod to be scheduled and the load information of each node; then traverse each node, input the Pod information of the Pod to be scheduled and the load information of the current node into the scheduling evaluation model, and obtain the evaluation information of the Pod to be scheduled when it is scheduled to the current node output by the scheduling evaluation model; preferably, the Pod to be scheduled is scheduled based on the evaluation information corresponding to each node. The method introduces the load information of the current node, avoids the situation where the physical resources on the node are abnormal due to only considering the hardware resources, ensures the safe operation of the Kubernetes cluster, and improves the stability of the Kubernetes cluster. At the same time, the method is combined with the scheduling evaluation model, which can make the scheduling process of the Pod to be scheduled more objective, the scheduling effect is better, the scheduling efficiency is higher, and the full utilization of physical resources can be achieved.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on the drawings in the following description without creative work.

FIG1 is a flow chart of a physical resource scheduling method for Kubernetes provided by the present invention;

FIG2 is a schematic diagram of the structure of a scheduler provided by the present invention;

FIG. 3 is a schematic diagram of the structure of an electronic device provided by the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Currently, Kubernetes mainly uses the Kubernetes native solution and the improved solution of the Kubernetes native solution when scheduling physical resources.

For the native solution, it is mainly based on the resource usage of the Node in the current Kubernetes cluster. After a series of selection operations, a Node that can be used to deploy the current new Pod is obtained, and the new Pod is deployed to the Node.

The process of Kubernetes selecting a suitable Node for a Pod includes three stages: Node pre-selection, Node optimization, and Node selection.

The Node pre-selection phase uses a series of pre-selection rules to check each Node, filter out the Nodes that do not meet the conditions, and complete the Node pre-selection. The pre-selection rules in this phase are mainly some mandatory rule verifications, which are used to check whether the host can match the resources required by the Pod. If there is a rule that no host can meet, the Pod will be suspended until a host can meet it. The resources involved in the rule verification in the pre-selection phase are basically incompressible resources (such as memory and hard disk space), which generally cannot be recycled unless the Pod is exited.

The node optimization phase is to prioritize the pre-selected nodes in order to select the most suitable node to run the Pod object. Node optimization scores the conditions of each node through a series of priority functions, and then obtains the final score of a node through the preset weight. The optimization phase mainly involves some compressible resources, such as CPU cycles, Disk I/O bandwidth, etc. These resources can be limited and recycled, and the Pod can reduce the usage of these resources so as not to be killed.

Node selection is to select the Node with the highest priority from the priority sorting results to run the Pod. If there are more than one such Node, one is randomly selected. After the scheduling process binds the Pod to the selected Node, Kubernetes will start the Pod on the Node.

When the Kubernetes scheduler performs the initial node selection, the hardware resource judgment is based on static resources. The main steps are as follows:

1) Get the Pod information deployed on each Node.

2) Get the static configuration information of the deployed Pod.

3) Calculate the total resource usage of the node based on the Requests information in the Pod-related configuration.

4) The score of each Node is calculated based on the remaining resources of each Node, and a higher score is given to a Node with a higher remaining resource amount.

5) Perform a preliminary selection based on the Node score calculated in the previous step and the scores obtained by other preliminary selection functions.

When scheduling Pods to Nodes, this solution may cause a discrepancy between the Requests configuration used in static resource checks and the actual situation. Therefore, in a cluster that has been running for a period of time, the following situation may occur: According to the Requests configuration, a certain node is relatively idle. However, in fact, the Pods running on the node occupy a large amount of memory, and the memory resources of the operating system running on the node are relatively tight.

Due to the deviation between the above configuration and the actual operation, when a large number of Pods need to be scheduled in Kubernetes, it is easy for this type of seemingly idle Node to be judged as the preferred result due to the pre-selection method based on the Requests configuration, resulting in more Pods being scheduled to the Node, causing abnormalities in the actual system resources. In serious cases, the Node may even go offline abnormally, which has an adverse impact on the overall operation security of Kubernetes.

The improvement plan for the Kubernetes native solution is mainly based on the Kubernetes native solution, adding consideration to the current resource usage of the Node and the historical Pod scheduling information. Although this solution considers that the Node where the scheduled Pod is located is currently the best from the current scheduling moment, due to the uncertainty of resource usage for application operation, the resource usage of the scheduled application may change later, resulting in abnormal cluster stability.

To this end, an embodiment of the present invention provides a physical resource scheduling method based on Kubernetes.

FIG1 is a flow chart of a physical resource scheduling method based on Kubernetes provided in an embodiment of the present invention. As shown in FIG1 , the method includes:

S1, obtain the scheduling information in the Kubernetes cluster, where the scheduling information includes the Pod information of the Pod to be scheduled and the load information of each node;

S2, traversing each of the nodes, inputting the Pod information of the Pod to be scheduled and the load information of the current node into the scheduling evaluation model, and obtaining the evaluation information of the Pod to be scheduled when being scheduled to the current node output by the scheduling evaluation model;

S3, scheduling the Pod to be scheduled based on the evaluation information corresponding to each of the nodes;

The scheduling evaluation model is trained based on a training sample set and evaluation information labels corresponding to each training sample in the training sample set. The training samples include Pod information of Pod samples and load information of node samples to which the Pod samples are scheduled.

Specifically, the Kubernetes-based physical resource scheduling method provided in the embodiment of the present invention is executed by a Kubernetes scheduler, which can be configured in a server. The server can be a local server or a cloud server. The local server can specifically be a computer, etc., which is not specifically limited in the embodiment of the present invention.

First, execute step S1 to obtain the scheduling information in the Kubernetes cluster. The Kubernetes cluster includes multiple Pods and multiple nodes. Pod is the smallest basic unit created or deployed by Kubernetes. A Pod represents a working unit running on the Kubernetes cluster and can include multiple container processes. A node is a host that can host the Pod to run, and each node has a corresponding operating system.

Scheduling information may include Pod information of the Pod to be scheduled and load information of each node. The Pod to be scheduled refers to the Pod that needs to determine which node to schedule it to. The Pod information may include deployment information of the Pod to be scheduled, operation-related information, and resource usage information of the application running on the Pod to be scheduled at the historical running time. Each node refers to a node in the Kubernetes cluster, and the load information may include the name, IP address, CPU usage, number of CPU cores, memory capacity, disk usage, disk capacity, and I/O rate of the node.

After obtaining the scheduling information in the Kubernetes cluster, since it contains a lot of attribute data, some of which may be irrelevant to the determination of subsequent evaluation information or may even interfere with the determination of subsequent evaluation information, the scheduling information can be preprocessed. The preprocessing operation may include: deduplication, unique value data deletion, numerical missing value interpolation, categorical feature encoding, and standardization.

1) De-duplicate data operation. In the process of collecting scheduling information, there are cases of repeated crawling, and this part of the data needs to be deleted. If multiple duplicate values are found through identification, this part of the duplicate data is directly deleted. In addition, useless data such as IP addresses and node names in the scheduling information can also be deleted.

2) Unique value data deletion operation: Since unique value data are usually some id attributes, these attributes cannot describe the distribution law of the sample itself, so simply delete these data.

3) Interpolation operation of missing values of numerical type. For the numerical data in the scheduling information, if the distance of the numerical data is measurable, the average value of the valid value of the numerical data is used to interpolate the missing value; if the distance of the numerical data is not measurable, the mode of the valid value of the numerical data is used to interpolate the missing value. Multiple interpolation can also be used. Multiple interpolation assumes that the values to be interpolated are random. In practice, the values to be interpolated are usually estimated, and different noises are added to form multiple groups of optional interpolation values. The most appropriate interpolation value is selected according to a certain selection basis.

4) Categorical feature encoding operation. For categorical features in the scheduling information, the categorical features are unordered categorical variables and should be converted into matrices. For categories with less than 10, one-hot encoding is used; for categories with more than 10, frequency encoding is used.

5) Numerical standardization: In the embodiment of the present invention, a min-max standardization method may be used to transform the scheduling information into the interval [0, 1].

Then execute step S2, traverse each node in the Kubernetes cluster, take the node traversed at the current moment as the current node, input the Pod information of the Pod to be scheduled and the load information of the current node into the scheduling evaluation model, and obtain the evaluation information of the Pod to be scheduled when it is scheduled to the current node output by the scheduling evaluation model. The scheduling evaluation model is used to characterize the suitability of the Pod to be scheduled to the current node, and the obtained evaluation information is usually expressed in the form of a score. The higher the score, the more suitable it is, and the lower the score, the less suitable it is. The scheduling evaluation model can be a decision tree model based on machine learning, or it can be a neural network model of machine learning, which is not specifically limited here.

The scheduling evaluation model can be obtained by training the initial scheduling evaluation model through the training sample set and the evaluation information labels corresponding to each training sample in the training sample set, combined with the loss function. The adopted training sample set may include multiple training samples, each training sample includes the Pod information of a Pod sample and the load information of the node sample to which the Pod sample is scheduled. Each training sample corresponds to an evaluation information label, which is used to characterize the suitability of the Pod sample in the training sample to be scheduled to the current node. The evaluation information label can be obtained by manual labeling, or reference information such as the scheduling rules of the Pod sample can be introduced during manual labeling, which is not specifically limited here.

Before using the training sample set and the evaluation information labels corresponding to each training sample in the training sample set to train the initial scheduling evaluation model, each training sample can be preprocessed. The preprocessing operation can also include: duplicate data deletion operation, unique value data deletion operation, numerical missing value interpolation operation, categorical feature encoding operation and standardization operation. When performing standardization operations on each training sample, each training sample x ₁ ,x ₂ …x _n can be linearly transformed so that the linear transformation results _yi fall into the interval [0,1] respectively. The conversion function is as follows:

Among them, n is the number of training samples.

In step S2, the load information of each node in the Kubernetes cluster and the Pod information of the Pod to be scheduled can be used as inputs of the scheduling evaluation model, and the scheduling evaluation model is used to obtain the evaluation information of the Pod to be scheduled being scheduled to each node in the Kubernetes cluster.

Finally, step S3 is executed to schedule the Pod to be scheduled according to the evaluation information corresponding to each node. Here, the evaluation information corresponding to each node is the evaluation information of the Pod to be scheduled to each node. The Pod to be scheduled can be directly scheduled to the node with the highest evaluation information, that is, the Pod to be scheduled is deployed to the node with the highest evaluation information.

The physical resource scheduling method based on Kubernetes provided in the embodiment of the present invention first obtains the scheduling information in the Kubernetes cluster, and the scheduling information includes the Pod information of the Pod to be scheduled and the load information of each node; then traverses each node, and inputs the Pod information of the Pod to be scheduled and the load information of the current node into the scheduling evaluation model, and obtains the evaluation information of the Pod to be scheduled when it is scheduled to the current node output by the scheduling evaluation model; preferably, the Pod to be scheduled is scheduled based on the evaluation information corresponding to each node. This method introduces the load information of the current node, avoids the situation where the physical resources on the node are abnormal due to only considering hardware resources, ensures the safe operation of the Kubernetes cluster, and improves the stability of the Kubernetes cluster. At the same time, this method combines the scheduling evaluation model, which can make the scheduling process of the Pod to be scheduled more objective, the scheduling effect is better, the scheduling efficiency is higher, and the full utilization of physical resources can be achieved.

During the scheduling process, the node's adaptability is predicted and evaluated using the scheduling evaluation model, and then the most suitable node is selected to deploy the Pod based on the evaluation information. Not only is scheduling based on the current system situation, but the compatibility of the pod and the node in the future can also be predicted to obtain the best scheduling result.

On the basis of the above-mentioned embodiment, in the physical resource scheduling method based on Kubernetes provided in the embodiment of the present invention, the evaluation information label corresponding to each training sample is determined based on the following method:

Based on the density clustering algorithm, cluster analysis is performed on each of the training samples to obtain multiple clusters;

Based on the sample features in the multiple clusters, the training samples included in the multiple clusters are collectively labeled to obtain evaluation information labels corresponding to the training samples.

Specifically, in an embodiment of the present invention, when determining the evaluation information label corresponding to each training sample, each training sample can be first clustered by a density clustering algorithm to obtain multiple clusters. The density clustering algorithm refers to a density-based clustering algorithm, which is an algorithm that performs clustering based on the density of the data set in spatial distribution. The density clustering algorithm can include a density-based spatial clustering application with noise (Density-Based Spatial Clustering of Application with Noise, DBSCAN) algorithm and an ordering point to identify the clustering structure (Ordering Points to identify the clustering structure, OPTICS) algorithm, etc.

The process of clustering analysis on each training sample is the process of classifying each training sample. Several training samples belonging to the same category constitute a cluster. Therefore, multiple clusters can be obtained after clustering analysis on each training sample.

Through cluster analysis, we can find the allocation rules of each node sample during resource scheduling. The scheduling relationship between Pod samples and node samples in the same cluster is similar to each other, while the scheduling relationship between Pod samples and node samples in different clusters is less similar.

Then, the features of the training samples in each cluster can be extracted to obtain the sample features in each cluster, and then the training samples contained in each cluster can be collectively labeled through the sample features in each cluster. That is, it can be understood that since all the training samples contained in each cluster belong to the same category, they can all be labeled with the same evaluation information label, so there is no need to manually label each training sample in the training sample set one by one, which can improve the labeling efficiency.

The centralized labeling process can be achieved manually, and the sample features in the clusters provide a basis for manual labeling. Manual labeling after obtaining multiple clusters is different from traditional manual labeling. Labelers can label each cluster centrally based on the multiple clusters obtained by cluster analysis. Since the allocation characteristics of each cluster are similar, the scheduling of each node in the cluster is also similar, and the evaluation information labels are also similar. This sets a better reference system for the evaluation information labels, eliminating the need for labelers to review each item one by one, greatly saving labeling time.

The content of the centralized labeling can be the node number of the Node to which each Pod is most suitable for scheduling. If there are N nodes in the Kubernetes cluster, the evaluation information labels can be represented as [1, 2, 3, ..., N] respectively.

In the embodiment of the present invention, a density clustering algorithm is used to perform cluster analysis on each training sample to obtain multiple clusters, and the Pod resource scheduling rules are more intuitively mined. Through the sample features in multiple clusters, the training samples contained in the multiple clusters are centrally labeled, which can improve the labeling efficiency of the training samples, save labeling time, and reduce labor costs.

On the basis of the above embodiment, the physical resource scheduling method based on Kubernetes provided in the embodiment of the present invention, based on the density clustering algorithm, performs cluster analysis on each of the training samples to obtain multiple cluster clusters, including:

S4, traversing each of the training samples, and judging whether the current training sample is a core sample based on the number of other training samples existing in a preset neighborhood of the current training sample; if the current training sample is a core sample, storing the current training sample in a core sample set;

S5, traversing each core sample in the core sample set, and determining whether other training samples existing in the preset neighborhood of the current core sample and the current core sample belong to the same initial cluster based on the degree of membership between the other training samples existing in the preset neighborhood of the current core sample and the current core sample;

S6, after determining the initial clusters corresponding to the nuclear samples in the nuclear sample set, calculating the silhouette coefficients between the initial clusters corresponding to the nuclear samples;

S7, adjusting preset parameters in the density clustering algorithm, repeatedly performing the traversal action, calculating a plurality of silhouette coefficients, and determining the plurality of clustering clusters based on the plurality of silhouette coefficients.

Specifically, in the embodiment of the present invention, when clustering analysis is performed on each training sample by a density clustering algorithm to obtain multiple clusters, the density clustering algorithm used is the DBSCAN algorithm. First, step S4 is executed, that is, each training sample is traversed, and the training sample traversed at the current moment is used as the current training sample. By calculating the distance between the current training sample and other training samples in each training sample except the current training sample, the number of other training samples existing in the preset neighborhood of the current training sample is determined. The preset radius e of the preset neighborhood can be set as needed and is not specifically limited here.

It is understandable that when calculating the distance between the current training sample and other training samples in each training sample, the Euclidean distance can be used. In addition, since the Euclidean distance calculation is relatively simple, it often cannot represent the true distance between each training sample, especially in the scenario where there are multiple training samples in the embodiment of the present invention, the differences between the training samples are large, and the Euclidean distance cannot fully express their distance characteristics. Therefore, in order to solve the problems caused by the Euclidean distance in the embodiment of the present invention, the Hausdorff distance can also be used to calculate, so as to improve the problem of incomplete distance measurement in the original DBSCAN algorithm.

Thereafter, it is determined whether the current training sample is a core sample based on the number of other training samples existing in the preset neighborhood of the current training sample. In the judgment process, a quantity threshold MinPts may be introduced, and then the relationship between the number of other training samples existing in the preset neighborhood of the current training sample and the quantity threshold is compared. If the number of other training samples existing in the preset neighborhood of the current training sample is greater than or equal to the quantity threshold, it can be determined that the current training sample is a core sample, otherwise it can be determined that the current training sample is not a core sample. It is understandable that the core sample may be the cluster center of the cluster analysis, that is, the center of each cluster cluster obtained subsequently.

When the current training sample is determined to be a core sample, the current training sample can be stored in the core sample set. After traversing each training sample, training samples belonging to the core sample in each training sample can be obtained, and all the core samples are stored in the core sample set.

Then, step S5 is executed to traverse each core sample in the core sample set, take the core sample traversed at the current moment as the current core sample, and calculate the membership between other training samples existing in the preset neighborhood of the current core sample and the current core sample.

Since the original DBSCAN algorithm treats each training sample equally, there may be noise points in the actual training samples. These noise points may affect the correct construction of each cluster, thereby reducing the generalization ability of the cluster analysis. In the embodiments of the present invention, more attention should be paid to these noise samples to distinguish the noise samples from the training samples in other clusters, so that subsequent labeling personnel can better distinguish the differences in scheduling relationships in the training samples and improve labeling efficiency.

Due to the overlap between clusters, some training samples distributed at the edge of the cluster do not necessarily belong to a certain cluster in practical applications, but may belong to multiple clusters or be noise samples, which leads to fuzzy uncertainty in the categories of these noise samples; while the certainty of the categories of training samples distributed in the center of the cluster is much higher than that of training samples distributed at the edge, so all training samples in the training sample set cannot be treated equally during clustering. Considering that the membership function is based on fuzzy set theory, unlike traditional clustering algorithms that simply judge whether a training sample belongs to a cluster, it can calculate the extent to which a training sample belongs to a cluster, so that the weight of each training sample can be controlled. Therefore, in order to eliminate the influence of noise samples, the membership function is introduced in the DBSCAN algorithm.

The membership function can be selected as needed. The traditional distance-based membership function can be selected, but this membership function only considers the relationship between the training sample and the center point of the category where the training sample is located, that is, the core sample, and does not consider the impact of the sample distribution on the membership. In the Kubernetes cluster, according to the original node scheduling characteristics and experience of the cluster, the real data distribution is dispersed. On this basis, the membership function can also be improved to take into account the dispersed distribution characteristics of the Pod sample and obtain the KNN membership function. The membership is calculated by the KNN membership function to improve the noise resistance of the clustering analysis algorithm.

Thereafter, by the degree of membership between other training samples existing in the preset neighborhood of the current core sample and the current core sample, it is determined whether other training samples existing in the preset neighborhood of the current core sample and the current core sample belong to the same initial cluster. When judging whether other training samples existing in the preset neighborhood of the current core sample and the current core sample belong to the same initial cluster, a membership threshold can be introduced, and then the degree of membership between each other training sample existing in the preset neighborhood of the current core sample and the current core sample is compared with the membership threshold. If the degree of membership between a certain other training sample existing in the preset neighborhood of the current core sample and the current core sample is less than the membership threshold, it can be determined that the other training sample and the current training sample do not belong to the same initial cluster. Otherwise, it can be determined that the other training sample and the current training sample belong to the same initial cluster, and the other training sample and the current training sample are stored in the same initial cluster together.

Then, step S6 is performed to determine the initial clusters corresponding to each core sample in the core sample set, and then calculate the silhouette coefficients between the initial clusters corresponding to each core sample. Since the preset parameters such as the preset radius e, the quantity threshold MinPts, and the membership threshold of the preset neighborhood in the initially determined density clustering algorithm may not necessarily guarantee the acquisition of appropriate clusters, the silhouette coefficients between the initial clusters corresponding to each core sample can be calculated so as to adjust the silhouette coefficients to obtain the preset radius and quantity threshold that optimize the clustering effect.

Here, the silhouette coefficient can be expressed by the following calculation formula:

Where _{ai is} the average distance between training sample i and all other training samples in its initial cluster, and _bi is the minimum distance between training sample i and all training samples in other initial clusters.

Each training sample corresponds to a silhouette coefficient, and the value range of the silhouette coefficient SC is between -1 and 1. When SC takes a negative value, the distance from training sample i to the training samples in other initial clusters is smaller than the distance from it to all other training samples in its own initial cluster, indicating that the two initial clusters overlap, and the clustering effect is poor. The larger the SC value, the higher the clustering quality and the better the clustering effect.

Finally, step S7 is executed to adjust the preset parameters in the density clustering algorithm, and the traversal action is repeated, that is, the above steps S4-S6 are repeated, and multiple silhouette coefficients are calculated. Multiple clusters are determined through multiple silhouette coefficients.

Here, the preset parameters may include parameters such as a preset radius e of a preset neighborhood, a quantity threshold MinPts, and a membership threshold.

When multiple clusters are determined by multiple silhouette coefficients, initial clusters corresponding to the largest silhouette coefficient can be selected from the multiple silhouette coefficients and used as the final cluster analysis results, that is, the initial clusters corresponding to the largest silhouette coefficient are used as the final clusters.

In the embodiment of the present invention, the membership degree and the silhouette coefficient are used to adjust the preset parameters, and then the optimal parameters are automatically selected, thereby solving the defect of inaccurate manual parameter setting in the traditional DBSCAN algorithm.

On the basis of the above-mentioned embodiment, in the Kubernetes-based physical resource scheduling method provided in the embodiment of the present invention, other training samples existing in the preset neighborhood of the current training sample are determined based on the following method:

Calculating the Hausdorff distance between the current training sample and each of the training samples;

If the Hausdorff distance is smaller than the preset radius of the preset neighborhood, it is determined that the training sample corresponding to the Hausdorff distance is other training samples existing in the preset neighborhood of the current training sample.

Specifically, in an embodiment of the present invention, when determining other training samples existing in a preset neighborhood of the current training sample, this can be achieved by calculating the Hausdorff distance between the current training sample and each training sample, which can represent the true distance between each training sample and make the distance metric more comprehensive.

After calculating the Hausdorff distance between the current training sample and each training sample, it can be determined whether the Hausdorff distance between each training sample and the current training sample is less than a preset radius. If it is less than the preset radius, it can be determined that the training sample is within the preset neighborhood of the current training sample, that is, other training samples within the preset neighborhood except the current training sample.

In the embodiment of the present invention, the Hausdorff distance is used to better measure the similarity between training samples and improve the robustness of the density clustering algorithm.

On the basis of the above-mentioned embodiment, in the Kubernetes-based physical resource scheduling method provided in the embodiment of the present invention, the degree of membership between other training samples existing in the preset neighborhood of the current core sample and the current core sample is determined based on the following method:

Traversing each other training sample existing in a preset neighborhood of the current core sample, and determining a number of adjacent samples that are adjacent to the other current training samples and are within the preset neighborhood of the current core sample;

The degree of membership between the current other training sample and the current core sample is determined based on the average, maximum and minimum values of the distances between the current other training sample and the plurality of adjacent samples.

Specifically, in an embodiment of the present invention, when determining the membership, although the KNN-based membership function can take into account the spatial distribution relationship of the training samples, it only takes the distances of several surrounding training sample points into consideration, and fails to reflect some special cases or overall situations. Therefore, a fuzzy judgment membership calculation method is proposed in an embodiment of the present invention, namely, the K nearest neighbor membership calculation method, which can reduce the impact of noise samples.

First, traverse each other training sample existing in the preset neighborhood of the current core sample, and take the other training sample existing in the preset neighborhood of the current core sample traversed at the current moment as the current other training sample. Determine a number of adjacent samples that are adjacent to the current other training sample and are in the preset neighborhood of the current core sample, that is, for the current other training sample x _i , find the k adjacent samples that are closest to it and are in the same category, forming a set D _i ={d ₁ ,d ₂ ,...,d _k }. Among them, d _j (j=1,2,...,k) is the distance from the current other training sample x _i to the jth adjacent sample, and k is the number of neighbors, which can be set as needed and is not specifically limited here.

It is understandable that the number of nearest neighbors k is also one of the preset parameters in the density clustering algorithm. Therefore, after calculating the silhouette coefficient between the initial clusters corresponding to each core sample in each round, it is necessary not only to adjust the preset radius e, the number threshold MinPts, and the membership threshold and other parameters, but also to adjust the number of nearest neighbors k.

Then, the membership between the current other training samples and the current core sample is determined by the mean, maximum and minimum values of the distances between the current other training samples and each adjacent sample. That is, the membership of the current other training samples x _i can be calculated by the following formula:

Among them, _dai is the average distance from the current other training samples _xi to the set _Di , _dmax and _dmin are the maximum and minimum values in _dai respectively; θ is the parameter used to control the lower limit of membership, θ<1 and is a sufficiently small positive number; f is the parameter that controls the changing trend of the membership function, which is a constant.

Thereafter, after traversing every other training sample in the preset neighborhood of the current kernel sample, the noise sample can be re-judged to determine whether the distance from the noise sample to each training sample in the generated initial cluster is less than or equal to the preset radius of the preset neighborhood. If so, the noise sample is classified into the initial cluster.

In the embodiment of the present invention, the K nearest neighbor membership is used to consider the knowledge in the fuzzy set and fully consider the influence of the surrounding sample points on the sample, which can effectively improve the accuracy of DBSCAN.

Based on the above embodiment, the improved DBSCAN clustering algorithm used in the embodiment of the present invention has the following process:

Input: training sample set D, preset radius ε, quantity threshold MinPts, membership threshold F, number of neighbors K.

Output: Cluster analysis results and noise samples

1) Randomly select an unprocessed training sample from the training sample set D;

2) The distance between the extracted training sample and other training samples in the training sample set D is calculated based on the Hausdorff distance. If the number of other training samples in the preset neighborhood of the extracted training sample meets the quantity threshold requirement, then it is a core sample. The core sample is placed in the core sample set seed and the sample point is classified as the initial cluster C. Otherwise, return to step 1);

3) Traverse each core sample in the seed, and determine whether the noise sample can be merged into the existing initial cluster C by calculating the K nearest neighbor membership between each sample point;

4) Traverse the entire training sample set D and perform DBSCAN clustering;

5) Calculate the silhouette coefficient SC after clustering and adjust preset parameters such as the preset radius ε, quantity threshold MinPts, membership threshold F, and number of neighbors K;

6) Repeat steps 4) and 5) and compare the calculated SC;

7) Selecting each initial cluster that satisfies the optimal SC in step 6) as the output cluster analysis result, that is, obtaining each cluster cluster.

On the basis of the above embodiment, the improved DBSCAN clustering algorithm adopted in the embodiment of the present invention, the scheduling evaluation model is trained based on the following method:

Based on the training samples and the evaluation information labels corresponding to each training sample in the training sample set, a loss function including a regularization term is used to iteratively train multiple classification decision tree models to obtain the scheduling evaluation model.

Specifically, in an embodiment of the present invention, the scheduling evaluation model used can be obtained by iteratively training multiple classification decision tree models using a loss function including a regularization term through a training sample set and an evaluation information label corresponding to each training sample. The classification decision tree model can be a CART classification decision tree model.

Here, the Boosting idea is adopted in the iterative training of the model. A series of CART classification decision tree models are used as weak learners (with low fitting degree), and multiple individual learners are trained based on the negative gradient of the loss function. They are integrated according to the gradient boosting method to form an accurate and reliable strong learner (with fitting degree close to the true value).

The iterative model training starts with a constant prediction. Each iteration generates a CART classification decision tree to fit the residual of the existing model, and the accuracy of the existing model is improved through multiple rounds of iterations. In addition, considering the problems of imbalanced categories and high dimensions of the labeled training sample set, a regularization term is added to the loss function during iterative model training to impose a penalty on the complexity of the model to improve the generalization ability of the model and prevent overfitting. The second-order derivative is used to make the loss function more accurate, and the idea of ensemble learning is used to optimize the problem of imbalanced categories.

Based on the above embodiments, in the Kubernetes-based physical resource scheduling method provided in the embodiments of the present invention, during the iterative training process, the loss function used in the current round of iterative training is determined based on the first-order gradient and second-order gradient of the loss function used in the previous round of iterative training of the current round.

Specifically, in the embodiment of the present invention, during the iterative training, assuming that the training sample set includes a training samples and b features, the training sample set can be expressed _as D = {( _xi , _yi )} ^, _xi∈Rb , _yi∈R.xi represents the feature of the ith training sample in the training sample set, _andyi represents the evaluation information label corresponding to the ith training sample.

The CART classification decision tree model uses an additive model and a forward step-by-step algorithm. The forward step-by-step algorithm is to optimize simultaneously on the basis of superimposing a new model. Specifically, each superimposed model fits the residual generated after the previous model is fitted. From the perspective of the algorithm model, Boosting Tree is an additive model of decision trees, so the final prediction value of K trees is for:

F＝{f(x)＝ω _q(x) },q:R ^b →T,ω∈R ^T

Among them, each function _fk corresponds to an independent tree structure vector q and leaf weight ω, q points to the corresponding leaf label from the training sample, each leaf node of each CART tree corresponds to a continuous score value, that is, the weight, the score of the i-th node is ω; T is the number of leaf nodes; F is the set of CART classification decision tree models; _ωq(x) is the predicted value obtained by each CART classification decision tree model. For each training sample, each CART classification decision tree model classifies it into a leaf node according to different classification rules, and obtains the final predicted value by accumulating the score ω of the corresponding leaf.

The loss function of the model can be expressed as:

Among them, n is the number of training samples.

The prediction accuracy of the model is determined by the model's bias and variance. The loss function represents the model's bias. If you want to reduce the variance, you need to add a regularization term to the objective function to prevent overfitting. Therefore, in order to learn the set of functions in the model, the objective function of the model is defined as:

in, is a differentiable convex loss function used to measure the gap between the predicted value and the label value; Ω(f _k ) is a regularization term used to penalize the complexity of the model. This additional regularization term can smooth the weights of each leaf node to avoid overfitting. γ and λ are regularization term parameters that control the complexity of the model. The larger the parameter value, the less likely the model is to overfit. T represents the number of leaf nodes in the tree.

The model needs to obtain f _k for each tree. Since the algorithm is based on the Boosting Tree model, it is impossible to obtain the model of all trees at once. The only way is to obtain the tth tree through iterative training. First, define the initial boosting tree as Then the model obtained at the tth iteration is:

After t iterations and accumulation, the objective function is:

Then perform a second-order Taylor expansion on the objective function of each training step. Using the second-order information of Taylor expansion can make the gradient converge faster and more accurately, because the first-order derivative guides the direction of the gradient, and the second-order derivative guides how the gradient direction changes. The combination of the two can more accurately approximate the real loss function. The objective function after the second-order Taylor expansion is:

Among them, _gi and _hi are the first-order and second-order gradients of the loss function respectively. Removing the constant term, the simplified objective function of the t-th training can be obtained as:

The final output of the model is the accumulation of the output results of this series of weak learners. This training mode not only retains the optimal segmentation of the CART classification decision tree for feature selection, but also eliminates the disadvantage of the decision tree's easy overfitting through the cumulative approximation method. In the end, a scheduling evaluation model is obtained that fully fits the training data and has sufficient generalization ability.

By tuning the training results, the final scheduling evaluation model can successfully predict the node number of the most suitable Node to which the current Pod should be assigned. This prediction result can be used to continue to iteratively optimize the scheduler.

After the scheduling evaluation model training is completed, continue to collect the required data. And use it together with the original data as new training data to continuously train the scheduling evaluation model to enhance the prediction accuracy of the scheduling evaluation model.

In summary, the physical resource scheduling method based on Kubernetes provided in the embodiment of the present invention adopts an artificial intelligence algorithm. When Kubernetes schedules the Pod of an application, it no longer simply schedules it based on the static resource allocation situation, but combines the historical resource usage, the historical situation of the application where the Pod is located, the resource usage of the current system, and the future resource usage forecast to make a comprehensive evaluation and judgment to obtain an optimal result. Its advantages are:

1) Solve the centralized scheduling problem that may be caused by the static scheduling mechanism of the native scheduler.

2) Introduce artificial intelligence algorithms to conduct comprehensive node evaluation, comprehensively consider historical cluster resource usage and possible future usage speculation to make comprehensive judgments, and the scheduling results are more accurate.

3) Supports deployment using Kubernetes' service mechanism to prevent single point failure of the scheduler.

4) Use the native Kubernetes mechanism to introduce a new scheduler scheduling mechanism, without any invasive modification to Kubernetes itself, and the code is completely decoupled.

As shown in FIG. 2 , based on the above embodiment, an embodiment of the present invention provides a scheduler, including:

The information acquisition module 21 is used to obtain the scheduling information in the Kubernetes cluster, where the scheduling information includes the Pod information of the Pod to be scheduled and the load information of each node;

An evaluation module 22 is used to traverse each of the nodes, input the Pod information of the Pod to be scheduled and the load information of the current node into a scheduling evaluation model, and obtain the evaluation information of the Pod to be scheduled when it is scheduled to the current node output by the scheduling evaluation model;

A scheduling module 23, configured to schedule the Pod to be scheduled based on the evaluation information corresponding to each of the nodes;

The scheduling evaluation model is trained based on a training sample set and evaluation information labels corresponding to each training sample in the training sample set, and the training samples include Pod information of Pod samples and load information of node samples to which the Pod samples are scheduled.

On the basis of the above embodiment, the scheduler provided in the embodiment of the present invention includes a clustering module, which is used for:

Based on the density clustering algorithm, cluster analysis is performed on each of the training samples to obtain multiple clusters;

Based on the sample features in the multiple clusters, the training samples included in the multiple clusters are collectively labeled to obtain evaluation information labels corresponding to the training samples.

On the basis of the above embodiment, the clustering module of the scheduler provided in the embodiment of the present invention is used for:

Traversing each of the training samples, and judging whether the current training sample is a core sample based on the number of other training samples existing in a preset neighborhood of the current training sample; if the current training sample is a core sample, storing the current training sample in a core sample set;

Traversing each core sample in the core sample set, and determining whether other training samples existing in a preset neighborhood of the current core sample and the current core sample belong to the same initial cluster based on the degree of membership between the current core sample and other training samples existing in a preset neighborhood of the current core sample;

After determining the initial clusters corresponding to the nuclear samples in the nuclear sample set, calculating the silhouette coefficients between the initial clusters corresponding to the nuclear samples;

The preset parameters in the density clustering algorithm are adjusted, the traversal action is repeatedly performed, a plurality of silhouette coefficients are calculated, and the plurality of cluster clusters are determined based on the plurality of silhouette coefficients.

On the basis of the above embodiment, the scheduler provided in the embodiment of the present invention further includes a distance calculation module, which is used to:

Calculating the Hausdorff distance between the current training sample and each of the training samples;

If the Hausdorff distance is smaller than the preset radius of the preset neighborhood, it is determined that the training sample corresponding to the Hausdorff distance is other training samples existing in the preset neighborhood of the current training sample.

On the basis of the above embodiment, the scheduler provided in the embodiment of the present invention further includes a membership calculation module, which is used to:

Traversing each other training sample existing in a preset neighborhood of the current core sample, and determining a number of adjacent samples that are adjacent to the other current training samples and are within the preset neighborhood of the current core sample;

The degree of membership between the current other training sample and the current core sample is determined based on the average, maximum and minimum values of the distances between the current other training sample and the plurality of adjacent samples.

On the basis of the above embodiment, the scheduler provided in the embodiment of the present invention further includes a training module for:

Based on the training samples and the evaluation information labels corresponding to each training sample in the training sample set, a loss function including a regularization term is used to iteratively train multiple classification decision tree models to obtain the scheduling evaluation model.

Based on the above embodiments, in the scheduler provided in the embodiments of the present invention, during the iterative training process, the loss function used in the current round of iterative training is determined based on the first-order gradient and second-order gradient of the loss function used in the previous round of iterative training.

On the basis of the above embodiment, the scheduler provided in the embodiment of the present invention allows adding a custom optimization function in the optimization process when scheduling Pods. When performing the optimization, the function in the custom scheduler is called through HTTP to obtain the corresponding optimization result, and the optimization judgment is performed in combination with the preset weight of the optimization result.

The following are the tasks involved in Kubernetes to ensure that the scheduler can implement scheduling:

The Kubernetes API server provides a Watch mechanism for peripheral components to obtain various information in the Kubernetes cluster. Pod Cache uses this mechanism to obtain changes in Pods in the Kubernetes cluster, and records the data in memory after data sorting for use by the optimization function.

Therefore, before using the scheduling evaluation model for evaluation, it is necessary to obtain the real-time monitoring data required for node evaluation from the database.

Kubernetes has a cache module that is used to obtain real-time monitoring data through the Watch mechanism and store it in memory. A) The cache module stores the change information of Pods in the Watch cluster and provides a query interface. B) Start monitoring Pod changes.

a) When the program starts, a parameter is set to configure the location of the Kubernetes config file in the running environment.

b) Based on the information in the configuration file, a Kubernetes client is created and a SharedInformerFactory is generated through the client.

c) Call Core().V1().Pods() on the Factory to generate the corresponding Informer and start the Watch action.

d) Provide three functions PodAdd, PodUpdate, and PodDelete, which are used to process the information returned by watch.

The data structure of the cache module is defined as follows: a) Since the structure corev1.Pod used when the Watched change information is returned already contains relatively detailed Pod-related information, the structure is simply encapsulated and named PodInfo; b) HostPodCache is defined, which contains a map defined as map[types.UID]*PodInfo, which is used to store all valid Pod information on a Node. c) PodInfoCache is defined, which contains a map defined as map[string]*HostPodCache, which is used to store the Pod information of all Nodes in the cluster. The key value of the map is used to store the name of the Node, which is obtained from the Kubernetes cluster.

The creation and maintenance methods of cache information in the cache module are as follows: a) The program defines a globally unique PodInfoCache instance, which is valid throughout the entire program operation and continuously maintains the data in it. b) When the Watch action takes effect, the API Service will return all the Pod information in the current cluster in a centralized manner. The module uses the processing function to sort out the Pod information and store it in the previously defined PodInfoCache structure. c) When the Pod in the cluster changes, after receiving the information sent by the API Service, the Pod information is updated in the cached data structure by judging the change status of the Pod and using the corresponding processing function.

The format of the preferred function used by the scheduler when scheduling Pod is as follows, which conforms to the format requirements of the preferred function in the scheduler extension mechanism provided by Kubernetes: Func func(pod v1.Pod,nodes[]v1.Node)(*schedulerapi.HostPriorityList,error).

The implementation process of the above optimization function is as follows:

a) Generate a schedulerapi.HostPriorityList to save the returned results.

b) Obtain the current cluster static resource data, dynamic monitoring data, Pod cache data, and real-time load data of all nodes.

c) Traverse the incoming cluster Node list Nodes: input the current node information, the Pod information to be scheduled, the related application information and other collected information into the evaluation model after preprocessing, and obtain the evaluation result of the Node for the current Pod. And save it in the result set.

d) Returns a result set that contains the scoring results for each Node in the passed list.

Specifically, the functions of each module in the scheduler provided in the embodiment of the present invention correspond one-to-one to the operation flow of each step in the above-mentioned method embodiment, and the effects achieved are also consistent. Please refer to the above-mentioned embodiment for details, and no further description will be given in the embodiment of the present invention.

Figure 3 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 3, the electronic device may include: a processor (Processor) 310, a communication interface (Communications Interface) 320, a memory (Memory) 330 and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call the logic instructions in the memory 330 to execute the Kubernetes-based physical resource scheduling method provided in the above embodiments, the method comprising: obtaining scheduling information in the Kubernetes cluster, the scheduling information including the Pod information of the Pod to be scheduled and the load information of each node; traversing each of the nodes, inputting the Pod information of the Pod to be scheduled and the load information of the current node into the scheduling evaluation model, and obtaining the evaluation information of the Pod to be scheduled when it is scheduled to the current node output by the scheduling evaluation model; scheduling the Pod to be scheduled based on the evaluation information corresponding to each of the nodes; wherein the scheduling evaluation model is trained based on a training sample set and the evaluation information labels corresponding to each training sample in the training sample set, and the training sample includes the Pod information of the Pod sample and the load information of the node sample to which the Pod sample is scheduled.

In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the Kubernetes-based physical resource scheduling method provided in the above-mentioned embodiments, the method including: obtaining scheduling information in the Kubernetes cluster, the scheduling information including Pod information of the Pod to be scheduled and load information of each node; traversing each of the nodes, inputting the Pod information of the Pod to be scheduled and the load information of the current node into a scheduling evaluation model, and obtaining the evaluation information of the Pod to be scheduled when it is scheduled to the current node output by the scheduling evaluation model; scheduling the Pod to be scheduled based on the evaluation information corresponding to each of the nodes; wherein the scheduling evaluation model is trained based on a training sample set and evaluation information labels corresponding to each training sample in the training sample set, and the training sample includes Pod information of the Pod sample and load information of the node sample to which the Pod sample is scheduled.

On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the Kubernetes-based physical resource scheduling method provided in the above-mentioned embodiments, the method comprising: obtaining scheduling information in the Kubernetes cluster, the scheduling information comprising Pod information of the Pod to be scheduled and load information of each node; traversing each of the nodes, inputting the Pod information of the Pod to be scheduled and the load information of the current node into a scheduling evaluation model, and obtaining the evaluation information of the Pod to be scheduled when it is scheduled to the current node output by the scheduling evaluation model; scheduling the Pod to be scheduled based on the evaluation information corresponding to each of the nodes; wherein the scheduling evaluation model is trained based on a training sample set and evaluation information labels corresponding to each training sample in the training sample set, and the training samples include Pod information of the Pod sample and load information of the node sample to which the Pod sample is scheduled.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. The physical resource scheduling method based on the Kubernetes is characterized by comprising the following steps of:

acquiring scheduling information in a Kubernetes cluster, wherein the scheduling information comprises Pod information of Pod to be scheduled and load information of each node;

traversing each node, and inputting the Pod information of the Pod to be scheduled and the load information of the current node into a scheduling evaluation model to obtain evaluation information, which is output by the scheduling evaluation model, when the Pod to be scheduled is scheduled to the current node;

scheduling the Pod to be scheduled based on the evaluation information corresponding to each node;

the scheduling evaluation model is obtained by training based on a training sample set and evaluation information labels corresponding to training samples in the training sample set, wherein the training samples comprise Pod information of Pod samples and load information of node samples to which the Pod samples are scheduled.

2. The Kubernetes-based physical resource scheduling method of claim 1, wherein the evaluation information label corresponding to each training sample is determined based on the following method:

performing cluster analysis on each training sample based on a density clustering algorithm to obtain a plurality of clusters;

And based on sample characteristics in the plurality of clusters, carrying out centralized labeling on training samples contained in the plurality of clusters to obtain evaluation information labels corresponding to the training samples.

3. The Kubernetes-based physical resource scheduling method of claim 2, wherein the clustering analysis is performed on each training sample based on a density clustering algorithm to obtain a plurality of clusters, and the method comprises:

traversing each training sample, and judging whether the current training sample is a core sample or not based on the number of other training samples existing in a preset neighborhood of the current training sample; if the current training sample is a core sample, storing the current training sample into a core sample set;

traversing each core sample in the core sample set, and determining whether other training samples existing in a preset neighbor of a current core sample and the current core sample belong to the same initial cluster or not based on membership degrees between the other training samples existing in the preset neighbor of the current core sample and the current core sample;

after determining initial clusters corresponding to each core sample in the core sample set, calculating contour coefficients among the initial clusters corresponding to each core sample;

And adjusting preset parameters in the density clustering algorithm, repeatedly executing traversing actions, calculating to obtain a plurality of contour coefficients, and determining a plurality of clustering clusters based on the plurality of contour coefficients.

4. The Kubernetes-based physical resource scheduling method of claim 3, wherein the other training samples present in the preset neighborhood of the current training sample are determined based on the following method:

calculating Hausdorff distance between the current training sample and each training sample;

and if the Hausdorff distance is smaller than the preset radius of the preset neighborhood, determining that the training sample corresponding to the Hausdorff distance is other training samples existing in the preset neighborhood of the current training sample.

5. The Kubernetes-based physical resource scheduling method of claim 3, wherein the membership degree between the current core sample and other training samples existing in the preset neighborhood of the current core sample is determined based on the following method:

traversing each other training sample existing in the preset neighborhood of the current core sample, and determining a plurality of adjacent samples which are adjacent to the current other training samples and are in the preset neighborhood of the current core sample;

And determining the membership degree between the current other training samples and the current core samples based on the average value, the maximum value and the minimum value of the distances between the current other training samples and the adjacent samples.

6. The Kubernetes-based physical resource scheduling method of any of claims 1-5, wherein the scheduling evaluation model is trained based on:

and performing iterative training on a plurality of classification decision tree models by adopting a loss function containing a regular term based on the training samples and the evaluation information labels corresponding to the training samples in the training sample set to obtain the scheduling evaluation model.

7. The Kubernetes-based physical resource scheduling method of claim 6, wherein the loss function employed in the iterative training of the current round is determined based on a first order gradient and a second order gradient of the loss function employed in the iterative training of the previous round of the current round.

8. A scheduler, comprising:

the information acquisition module is used for acquiring scheduling information in the Kubernetes cluster, wherein the scheduling information comprises Pod information of Pod to be scheduled and load information of each node;

The evaluation module is used for traversing each node, inputting the Pod information of the Pod to be scheduled and the load information of the current node into a scheduling evaluation model, and obtaining the evaluation information, output by the scheduling evaluation model, of the Pod to be scheduled when the Pod to be scheduled is scheduled to the current node;

the scheduling module is used for scheduling the Pod to be scheduled based on the evaluation information corresponding to each node;

the scheduling evaluation model is obtained by training based on a training sample set and evaluation information labels corresponding to training samples in the training sample set, wherein the training samples comprise Pod information of Pod samples and load information of node samples to which the Pod samples are scheduled.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the Kubernetes-based physical resource scheduling method of any one of claims 1 to 7 when the program is executed by the processor.

10. A computer program product comprising a computer program which, when executed by a processor, implements a Kubernetes-based physical resource scheduling method according to any one of claims 1 to 7.