WO2022110444A1

WO2022110444A1 - Dynamic prediction method and apparatus for cloud native resources, computer device and storage medium

Info

Publication number: WO2022110444A1
Application number: PCT/CN2020/139679
Authority: WO
Inventors: 叶可江; 陈文艳; 须成忠
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2020-11-30
Filing date: 2020-12-25
Publication date: 2022-06-02
Also published as: CN112565378A

Abstract

A dynamic prediction method for cloud native resources, comprising: on the basis of a Pearson correlation coefficient, performing correlation sorting on obtained resource data to be predicted and performance index data to obtain a correlation relation between the resource data to be predicted and the performance index data (S3); defining a correlation threshold on the basis of the correlation relation (S4); using the performance index data greater than or equal to the correlation threshold as performance index time series data (S5); horizontally expanding the performance index time series data to obtain training data and test data (S6); inputting the training data into a constructed sequential neural network model to carry out training (S7), to obtain a trained sequential neural network prediction model; and inputting the test data into the sequential neural network prediction model to carry out prediction operation, to obtain resource prediction results (S8). Further provided are a dynamic prediction apparatus for cloud native resources, a computer device and a storage medium, capable of reducing prediction complexity and improving prediction accuracy.

Description

Cloud native resource dynamic prediction method, device, computer equipment and storage medium

technical field

The present application relates to the field of information technology, and in particular, to a method, apparatus, computer equipment and storage medium for dynamic prediction of cloud native resources.

Background technique

The rapid development of cloud-native technologies has led to a sharp increase in the number of users and data scale, which has brought severe problems and challenges to the resource management of cloud-native clusters. On the one hand, users’ resource requests are very frequent and diverse, and existing resource forecasts can only make accurate forecasts for traditional periodicity, but cannot accurately predict the emergence of mutation points; on the other hand, real-time online services Hybrid deployment with offline jobs improves the performance of cloud-native clusters to a certain extent, but this co-location mode also brings resource competition and performance degradation, further increasing the complexity of resource prediction. In addition, traditional resource prediction models often have a certain delay, which causes certain performance obstacles to real-time dynamic resource allocation and management. Therefore, how to make real-time and accurate prediction of resources in a hybrid cloud-native cluster, so as to dynamically allocate reasonable resource allocation to the load, is the key issue of current research.

As an emerging cloud computing model, cloud native has the characteristics of high scalability, on-demand access, and lighter weight. More and more enterprises and individuals choose to use cloud native platforms to provide services. However, due to the complexity, heterogeneity and dynamic characteristics of upper-layer cloud-native applications, the requirements for resource management are getting higher and higher. Therefore, in order to effectively improve the performance of cloud-native cluster resource management, mixed deployment of different types of applications The approach has been widely used in cloud-native platforms.

However, the current resource prediction methods are mainly based on linear regression methods and machine learning methods. Among them, the first method has good accuracy in predicting the periodicity of the model, but often cannot predict the mutation point very well. Moreover, such methods usually only consider the time-series correlation of the forecast resources, and easily ignore the impact of other performance indicators on the forecast resources; the second method trains and cross-validates historical data through a machine learning model, and can input multi-dimensional resource data. , and the long-term memory of neural network can effectively capture time series information, but this method still has many deficiencies in the prediction of mutation points.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to provide a method, device, computer equipment and storage medium for dynamic prediction of cloud native resources, so as to at least solve the problems of high prediction complexity and low prediction accuracy of mutation points in traditional resource prediction methods.

In order to solve the above technical problems, an embodiment of the present application provides a method for dynamic prediction of cloud native resources, which adopts the following technical solutions:

Receive the dynamic prediction request sent by the user terminal;

Respond to dynamic prediction requests, read the local database, and obtain the resource data to be predicted and the performance indicator data of the container load in the cloud native cluster;

Based on the Pearson correlation coefficient, the correlation degree of the resource data to be predicted and the performance index data is sorted, and the correlation between the resource data to be predicted and the performance index data is obtained;

Define relevance thresholds based on relevance relationships;

Use the performance index data greater than or equal to the correlation threshold as the performance index time series data;

Perform horizontal data expansion of performance index time series data to obtain training data and test data;

Input the training data into the constructed time series neural network model for training, and obtain the trained time series neural network prediction model;

Input the test data into the time series neural network prediction model for prediction operation to obtain the resource prediction result.

Further, the method also includes:

Collect historical resource data of container loads in cloud-native clusters based on preset time intervals;

The historical resource data is preprocessed to obtain the resource data to be predicted.

Further, the steps of preprocessing the historical resource data to obtain the resource data to be predicted include:

Delete invalid or abnormal data in historical resource data to obtain valid time series data;

The effective time series data is normalized to obtain the resource data to be predicted.

Further, the method also includes:

Repeat the prediction operation process to obtain real-time prediction data;

Feed back real-time prediction data to the user terminal.

Further, the method also includes:

Add the preset fully connected layer and attention mechanism to the basic model architecture of the temporal neural network to obtain the temporal neural network model.

In order to solve the above technical problems, the embodiments of the present application further provide a cloud native resource dynamic prediction device, which adopts the following technical solutions:

a request receiving module for receiving the dynamic prediction request sent by the user terminal;

The request response module is used to respond to the dynamic prediction request, read the local database, and obtain the resource data to be predicted and the performance indicator data of the container load in the cloud native cluster;

The correlation ranking module is used to sort the correlation degree of the resource data to be predicted and the performance index data based on the Pearson correlation coefficient, so as to obtain the correlation between the resource data to be predicted and the performance index data;

Threshold definition module, used to define the correlation threshold based on the correlation relationship;

The time series data acquisition module is used to use the performance index data greater than or equal to the correlation threshold as the performance index time series data;

The data expansion module is used for horizontal data expansion of performance index time series data to obtain training data and test data;

The model training module is used to input the training data into the constructed time series neural network model for training, and obtain the trained time series neural network prediction model;

The data prediction module is used to input the test data into the time series neural network prediction model for prediction operation, and obtain the resource prediction result.

Further, the device also includes:

The data collection module is used to collect historical resource data of container loads in the cloud native cluster based on a preset time interval;

The preprocessing module is used to preprocess the historical resource data to obtain the resource data to be predicted.

Further, the preprocessing module includes:

The data deletion unit is used to delete invalid or abnormal data in the historical resource data to obtain valid time series data;

The normalization processing unit is used for normalizing the valid time series data to obtain the resource data to be predicted.

In order to solve the above-mentioned technical problems, the embodiment of the present application also provides a computer device, which adopts the following technical solutions:

A memory and a processor are included, a computer program is stored in the memory, and when the processor executes the computer program, the steps of the cloud native resource dynamic prediction method described above are implemented.

In order to solve the above technical problems, the embodiments of the present application also provide a computer-readable storage medium, which adopts the following technical solutions:

A computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, the steps of the cloud native resource dynamic prediction method described above are implemented.

Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects:

The present application provides a method for dynamic prediction of cloud native resources, including: receiving a dynamic prediction request sent by a user terminal; responding to the dynamic prediction request, reading a local database, and acquiring resource data to be predicted and performance indicator data of container loads in a cloud native cluster ; Based on the Pearson correlation coefficient, sort the correlation degree of the resource data to be predicted and the performance index data to obtain the correlation relationship between the resource data to be predicted and the performance indicator data; define the correlation threshold value based on the correlation relationship; it will be greater than or equal to the correlation degree The performance index data of the threshold is used as the performance index time series data; the performance index time series data is horizontally expanded to obtain training data and test data; the training data is input into the constructed time series neural network model for training, and the trained time series neural network is obtained. Network prediction model; input the test data into the time series neural network prediction model for prediction operation, and obtain the resource prediction result. Based on the Pearson correlation coefficient, the correlation degree of the resource data to be predicted and the performance index data obtained in the local database is sorted, so as to obtain the correlation relationship between the resource data to be predicted and the performance index data to define the correlation threshold. And based on the correlation threshold to obtain the performance index time series data; then based on the horizontal data expansion, the performance index time series data is pruned and information extracted to obtain training data and test data, which can achieve data retention on the basis of reducing input data. Then, based on the training data, a time-series neural network prediction model that can be used to capture long-term dependent time-series data information is trained, and then the test data is used as the input of the time-series neural network prediction model, and a prediction model with high prediction accuracy is obtained. Resource forecast results. By reducing the input data of the time series neural network model, the computational complexity can be effectively reduced, thereby reducing the prediction complexity, and to a certain extent, improving the accuracy and efficiency of cloud native cluster resource prediction.

Description of drawings

In order to illustrate the solutions in the present application more clearly, the following will briefly introduce the accompanying drawings used in the description of the embodiments of the present application. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a schematic diagram of an exemplary principle to which the present application can be applied;

FIG. 2 is a flowchart of an embodiment of a cloud native resource dynamic prediction method according to the present application;

3 is a flowchart of data preprocessing according to the cloud native resource dynamic prediction method of the present application;

Fig. 4 is a flow chart of a specific implementation manner of step S302 in Fig. 3;

5 is a schematic structural diagram of an embodiment of a cloud native resource dynamic prediction apparatus according to the present application;

6 is a schematic structural diagram of data preprocessing of the cloud native resource dynamic prediction device according to the present application;

7 is a schematic structural diagram of a specific implementation of the preprocessing module in FIG. 6;

FIG. 8 is a schematic structural diagram of an embodiment of a computer device according to the present application.

Detailed ways

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field of this application; the terms used herein in the specification of the application are for the purpose of describing specific embodiments only It is not intended to limit the application; the terms "comprising" and "having" and any variations thereof in the description and claims of this application and the above description of the drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings.

Example 1

Referring to FIG. 1 and FIG. 2 , a flowchart of an embodiment of a method for dynamic prediction of cloud native resources provided according to Embodiment 1 of the present application is shown. For the convenience of description, only parts related to the present application are shown.

In step S1, a dynamic prediction request sent by the user terminal is received.

In this embodiment, in this embodiment, the dynamic prediction request is for the user to have an in-depth understanding of the characteristics of cloud-native cluster resources, so as to select an appropriate resource prediction model and optimize it in combination with a specific scenario, so as to provide effective resources for dynamic resource allocation. The action request issued based on the decision of the value.

In step S2, in response to the dynamic prediction request, the local database is read, and the resource data to be predicted and the performance indicator data of the container load in the cloud native cluster are obtained.

In this embodiment, the resource data to be predicted refers to the time series data obtained by processing the historical resource data of the container load in the cloud native cluster according to a preset processing method, wherein the processing method may specifically be reasonable compression or extraction. There are no specific restrictions here, which can reduce the size of the input data without missing long-term dependency information, which is beneficial to improve the training speed of the subsequent training time series neural network prediction model, so as to realize real-time and dynamic resource management. It can reduce the lag of resource allocation, thereby improving the accuracy and efficiency of cloud native cluster resource prediction to a certain extent.

In this embodiment, the performance indicator data may specifically be performance indicators of the application layer such as CPU utilization, memory utilization, disk IO size, network bandwidth, etc.; and performance indicators of the microarchitecture layer such as IPC (Instructions per Cycle), branch Prediction (Branch Predict) and Cache misses (Cache misses) can be used to intuitively reflect the indicator data of cluster performance.

In this embodiment, since the resource utilization of different applications has the characteristics of dynamics and complexity, this embodiment obtains the load of containers in the cloud native cluster based on the preset processing method in the local database based on the dynamic prediction request. The historical resource data is processed according to the time series data, that is, the resource data to be predicted, and the performance index data that can be used to intuitively reflect the cluster performance, so that the subsequent resource data and performance index data based on the to-be-predicted resource data and performance index data can not be missing. Reducing the size of input data under the premise of long-term dependence on information is conducive to improving the training speed of the subsequent training time series neural network prediction model, so as to realize real-time and dynamic allocation of resources, reduce the lag of resource allocation, and improve to a certain extent. The accuracy and efficiency of cloud-native cluster resource prediction.

In step S3, the correlation degree of the resource data to be predicted and the performance index data is sorted based on the Pearson correlation coefficient, and the correlation relationship between the resource data to be predicted and the performance index data is obtained.

In this embodiment, the Pearson correlation coefficient is a coefficient capable of measuring the correlation strength of the resource data to be predicted and the performance index data.

Among them, the Pearson correlation coefficient is expressed as:

The resource data to be predicted is r, X represents the time series data of r, Y represents the time series data of other performance index data, and n represents the length of the time series data.

In this embodiment, the correlation relationship refers to the strong and weak correlation relationship between different performance index data and the resource data r to be predicted, respectively.

In this embodiment, due to the mixed deployment of different loads, resource competition will occur due to limited resources at the same time, and the degree of this competition is closely related to the load type. Therefore, this embodiment is based on the Pearson correlation coefficient. Calculate the correlation coefficients of the performance index data to obtain the correlation coefficients between the different performance index data and the resource data r to be predicted, and then sort these coefficients according to a preset sorting method, and based on the sorted correlation coefficients to represent the strong and weak correlation between different performance index data and the resource data r to be predicted, so that the subsequent pruning of the input data can be further realized based on the strong and weak relationship, so as to reduce the basis of input data. To a certain extent, the accuracy and efficiency of cloud-native cluster resource prediction can be improved.

In step S4, a correlation threshold is defined based on the correlation relationship.

In this embodiment, the correlation threshold is an index used to extract time series data with strong correlation, and the self-defined threshold Cmax based on the strong and weak relationship of the correlation, that is, the correlation threshold, can further realize the pruning of the input data, Therefore, the effective information of the data is retained on the basis of reducing the input data, thereby improving the accuracy and efficiency of cloud-native cluster resource prediction to a certain extent.

In step S5, the performance index data greater than or equal to the correlation threshold is used as the performance index time series data.

In this embodiment, based on a self-defined correlation threshold based on the strong and weak relationship of the correlation, all performance index data smaller than the correlation threshold can be deleted, and the performance index data greater than or equal to Cmax is retained as the performance index time series data, In order to enable the subsequent pruning of the input data based on the time series data of the performance index, the effective information of the data can be retained on the basis of reducing the input data, thereby improving the accuracy and efficiency of cloud-native cluster resource prediction to a certain extent.

In step S6, horizontal data expansion is performed on the performance index time series data to obtain training data and test data.

In this embodiment, horizontal data expansion is performed on the performance index time series data, specifically, by assuming that the predicted performance index data is cpu, and the correlation value greater than Cmax is the performance index data is cpu, memory, disk, then the data at time t The input matrix is arr=[cpu _t ,memory _t ,disk _t ], and the expanded input matrix is arr=[cpu _t-2 ,cpu _t-1 ,cpu _t ,memory _t-2 ,memory _t-1 ,memory _t ,disk _t-2 ,disk _t-1 ,disk _t ].

In this embodiment, the training data and the test data are obtained by performing horizontal data expansion and expansion based on the performance index time series data. Data and test data that can be used for resource prediction can realize the pruning of input data, so as to retain the effective information of the data on the basis of reducing the input data, thereby improving the accuracy of cloud-native cluster resource prediction to a certain extent. Efficiency.

In step S7, the training data is input into the constructed time series neural network model for training, and the trained time series neural network prediction model is obtained.

In this embodiment, the time-series neural network prediction model is optimized based on time-series neural networks (TCNs). By using dilated convolution and attention mechanisms, a wider receptive field can be obtained with less data, and a longer The time-dependent time-series data information capture model can effectively retain long-term dependent information, thereby effectively improving the accuracy of cloud native resource prediction to a certain extent.

In this embodiment, the training data that can be used for training the time-series neural network model is input into the constructed time-series neural network model for training. Less data gets a wider receptive field, and the long-term dependent time series data information is captured by the time series neural network prediction model, which can effectively retain the long-term dependency information, thereby effectively improving the accuracy of cloud native resource prediction to a certain extent.

In step S8, the test data is input into the time-series neural network prediction model for prediction operation to obtain the resource prediction result.

In this embodiment, after the training of the time-series neural network prediction model is completed, the test data that can be used for resource prediction is directly input into the trained time-series neural network prediction model to predict cloud native resources. Resource forecast results for predicted future resource utilization.

Continuing to refer to FIG. 3 , a flow chart of data preprocessing of the cloud native resource dynamic prediction method provided in Embodiment 1 of the present application is shown. For convenience of description, only parts related to the present application are shown.

In some optional implementations of the first embodiment, before step S2 responds to the dynamic prediction request, reads the local database, and obtains resource data to be predicted and performance indicator data of the container load in the cloud native cluster, the method further includes: Step S301 and Step S302.

In step S301, historical resource data of container loads in the cloud native cluster is collected based on a preset time interval.

In step S302, the historical resource data is preprocessed to obtain resource data to be predicted.

In this embodiment, the historical resource data may specifically include attribute values such as CPU utilization, memory utilization, disk IO size, and network bandwidth, and the sampling frequency is data such as once every 60s.

In this embodiment, the historical resource data is preprocessed, specifically, the resource data to be predicted can be obtained by deleting invalid and abnormal data.

In this embodiment, according to a preset time interval, such as every 60s, the container load in the cloud native cluster includes attribute values such as CPU utilization, memory utilization, disk IO size, network bandwidth, etc., and the sampling frequency is data such as once every 60s. Collect the historical resource data, and then delete the invalid and abnormal data preprocessing operation to obtain the resource data to be predicted. The collected historical resource data can be reasonably compressed, extracted and format converted. Then, the input data size can be reduced without missing long-term dependency information, so that the subsequent training speed of the time series neural network model can be improved based on the reduction of input data, so as to realize real-time and dynamic allocation of resources and reduce resource allocation. Therefore, to a certain extent, the accuracy and efficiency of cloud-native cluster resource prediction can be improved.

Continuing to refer to FIG. 4 , a flowchart of a specific implementation manner of step S302 provided in Embodiment 1 of the present application is shown. For the convenience of description, only the part related to the present application is shown.

In some optional implementation manners of the first embodiment, step S302 preprocesses historical resource data, and the step of obtaining resource data to be predicted includes steps S401 and S402.

In step S401, invalid or abnormal data in the historical resource data is deleted to obtain valid time series data.

In step S402, the effective time series data is normalized to obtain resource data to be predicted.

In this embodiment, in order to effectively improve the performance of cloud-native cluster resource management, this embodiment improves prediction time by pruning input data, so as to reasonably allocate cloud-native resources to reduce resource oversale and resource oversale. Therefore, in order to realize the pruning of the input data, this embodiment first deletes invalid or abnormal data in the historical resource data, and then normalizes the valid time series data obtained after deletion. Reduction and effective extraction of effective time series data to obtain resource data to be predicted.

In some optional implementations of the first embodiment, after step S8, the method further includes:

Repeat the prediction operation process to obtain real-time prediction data;

Feed back real-time prediction data to the user terminal.

In this embodiment, the real-time prediction data is prediction data including the predicted future resource utilization rate obtained by dynamically online real-time prediction of resources based on a time-series neural network prediction model.

In this embodiment, in order to realize real-time and dynamic allocation of resources and reduce the hysteresis of resource allocation, this embodiment repeatedly performs a prediction operation process of inputting test data into a time-series neural network prediction model, so as to obtain information including Real-time forecast data of predicted future resource utilization to meet the real-time demand for resource allocation.

In some optional implementations of the first embodiment, before step S7, the method further includes:

In this embodiment, the basic model of the temporal neural network is temporal neural network TCNs (Temporal Convolutional Networks).

In this embodiment, a preset fully connected layer and an attention mechanism are added to the basic model architecture of the time series neural network. Specifically, a layer of fully connected layer and an attention mechanism can be added on the basis of the time series neural network TCNs, This enables the time-series neural network model to capture long-term dependent information with a small amount of data during training, so as to reasonably dynamically allocate resources and improve the performance of cloud-native resource management to a certain extent.

In summary, the present application provides a method for dynamic prediction of cloud native resources, including: receiving a dynamic prediction request sent by a user terminal; responding to the dynamic prediction request, reading a local database, and obtaining resources to be predicted for container loads in a cloud native cluster data and performance indicator data; sort the correlation degree of the resource data to be predicted and the performance indicator data based on the Pearson correlation coefficient, and obtain the correlation relationship between the resource data to be predicted and the performance indicator data; define the correlation threshold based on the correlation relationship; The performance index data that is greater than or equal to the correlation threshold is used as the performance index time series data; the performance index time series data is extended horizontally to obtain training data and test data; the training data is input into the constructed time series neural network model for training, and the result is obtained The trained time-series neural network prediction model; input the test data into the time-series neural network prediction model to perform prediction operations to obtain resource prediction results. Based on the preset time interval, the historical resource data of the container load in the cloud native cluster is collected, and the collected historical resource data is deleted, normalized and other preprocessing to obtain the resource data to be predicted; then, based on the Pearson correlation The coefficients are used to sort the relevancy of the resource data to be predicted and the performance indicator data to obtain the correlation between the resource data to be predicted and the performance indicator data to define the relevancy threshold, and obtain the performance indicator time series data based on the relevancy threshold ; and then perform pruning and information extraction on the performance index time series data based on horizontal data expansion to obtain training data and test data, which can retain the effective information of the data on the basis of reducing the input data; then, based on the training data, add The fully connected layer and the time series neural network model of the attention mechanism are trained to obtain a time series neural network prediction model that can be used to capture long-term dependent time series data information, and then the test data is used as the input of the time series neural network prediction model to obtain predictions. A resource prediction result with a higher accuracy rate; then, the prediction operation process is repeatedly performed, and the obtained real-time prediction data is fed back to the user terminal. By reducing the input data of the time series neural network model, the computational complexity can be effectively reduced, thereby reducing the prediction complexity, and to a certain extent, improving the accuracy and efficiency of cloud native cluster resource prediction.

Those of ordinary skill in the art can understand that the realization of all or part of the processes in the methods of the above embodiments can be accomplished by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the program is During execution, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the aforementioned storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

It should be understood that although the various steps in the flowchart of the accompanying drawings are sequentially shown in the order indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order and may be performed in other orders. Moreover, at least a part of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution sequence is also It does not have to be performed sequentially, but may be performed alternately or alternately with other steps or at least a portion of sub-steps or stages of other steps.

Embodiment 2

With further reference to FIG. 5 , as an implementation of the method shown in FIG. 2 above, the present application provides an embodiment of a cloud native resource dynamic prediction apparatus. The apparatus embodiment corresponds to the method embodiment shown in FIG. 2 . The device can be specifically applied to various electronic devices.

As shown in FIG. 5 , the cloud native resource dynamic prediction apparatus 100 in this embodiment includes: a request receiving module 101, a request response module 102, a correlation ranking module 103, a threshold definition module 104, a time series data acquisition module 105, and a data expansion module. module 106 , model training module 107 and data prediction module 108 . in:

A request receiving module 101, configured to receive a dynamic prediction request sent by a user terminal;

The request response module 102 is configured to respond to the dynamic prediction request, read the local database, and obtain the resource data to be predicted and the performance indicator data of the container load in the cloud native cluster;

The correlation ranking module 103 is configured to perform correlation ranking on the resource data to be predicted and the performance index data based on the Pearson correlation coefficient, so as to obtain the correlation relationship between the resource data to be predicted and the performance index data;

Among them, the Pearson correlation coefficient is expressed as:

a threshold definition module 104, configured to define a correlation threshold based on the correlation relationship;

a time series data acquisition module 105, configured to use the performance index data greater than or equal to the correlation threshold as the performance index time series data;

The data expansion module 106 is used to perform horizontal data expansion on the performance index time series data to obtain training data and test data;

In this embodiment, horizontal data expansion is performed on the performance index time series data, specifically, by assuming that the predicted performance index data is cpu, and the correlation value greater than Cmax is the performance index data of cpu, memory, disk, then the data at time t The input matrix is arr=[cpu _t ,memory _t ,disk _t ], and the expanded input matrix is arr=[cpu _t-2 ,cpu _t-1 ,cpu _t ,memory _t-2 ,memory _t-1 ,memory _t ,disk _t-2 ,disk _t-1 ,disk _t ].

The model training module 107 is used to input the training data into the constructed time series neural network model for training, and obtain the trained time series neural network prediction model;

The data prediction module 108 is configured to input the test data into the time-series neural network prediction model to perform prediction operations to obtain resource prediction results.

The present application provides a cloud native resource dynamic prediction device, which includes: based on the Pearson correlation coefficient, the correlation degree of the resource data to be predicted and the performance index data obtained in the local database are sorted, so as to obtain the resource data to be predicted and the performance index. The correlation between the data is used to define the correlation threshold, and based on the correlation threshold, the performance index time series data is obtained; then based on the horizontal data expansion, the performance index time series data is pruned and information extracted to obtain training data and The test data can retain the effective information of the data on the basis of reducing the input data; then, based on the training data, a time-series neural network prediction model that can be used to capture long-term dependent time-series data information is trained, and then the test data is used as a model. The input of the time series neural network prediction model is used to obtain resource prediction results with high prediction accuracy. It can effectively reduce the computational complexity by reducing the input data of the time series neural network model, thereby reducing the forecasting complexity, and improving the accuracy and efficiency of cloud-native cluster resource forecasting to a certain extent.

Continuing to refer to FIG. 6 , a schematic structural diagram of the data preprocessing of the apparatus for dynamic prediction of cloud native resources provided in Embodiment 1 of the present application is shown. For convenience of description, only parts related to the present application are shown.

In some optional implementation manners of the second embodiment, the apparatus further includes: a data collection module 601 and a preprocessing module 602 .

A data collection module 601, configured to collect historical resource data of container loads in the cloud native cluster based on a preset time interval;

The preprocessing module 602 is used for preprocessing historical resource data to obtain resource data to be predicted.

In this embodiment, according to a preset time interval, such as every 60s, the container load in the cloud native cluster includes attribute values such as CPU utilization, memory utilization, disk IO size, network bandwidth, etc., and the sampling frequency is data such as once every 60s. Collect the historical resource data, and then delete the invalid and abnormal data of the historical resource data to obtain the resource data to be predicted. The collected historical resource data can be reasonably compressed, extracted and format converted. Then, the input data size can be reduced without missing long-term dependency information, so that the subsequent training speed of the time series neural network model can be improved based on the reduction of input data, so as to realize real-time and dynamic allocation of resources and reduce resource allocation. Therefore, to a certain extent, the accuracy and efficiency of cloud-native cluster resource prediction can be improved.

Continuing to refer to FIG. 7 , a flowchart of a specific implementation manner of the preprocessing module 602 in FIG. 6 provided in Embodiment 1 of the present application is shown. For the convenience of description, only parts related to the present application are shown.

In some optional implementations of the second embodiment, the preprocessing module 602 includes: a data deletion unit 701 and a normalization processing unit 702 .

A data deletion unit 701 is used to delete invalid or abnormal data in the historical resource data to obtain valid time series data;

The normalization processing unit 702 is configured to perform normalization processing on the valid time series data to obtain resource data to be predicted.

In some optional implementation manners of the second embodiment, the apparatus further includes: a real-time prediction module and a data feedback module.

The real-time prediction module is used to repeatedly perform the prediction operation process to obtain real-time prediction data;

The data feedback module is used to feed back the real-time prediction data to the user terminal.

In some optional implementation manners of the second embodiment, the device further includes:

To sum up, the present application provides a cloud native resource dynamic prediction device, including: a request receiving module for receiving a dynamic prediction request sent by a user terminal; a request response module for responding to the dynamic prediction request and reading a local database , to obtain the resource data to be predicted and the performance index data of the container load in the cloud native cluster; the correlation ranking module is used to sort the predicted resource data and the performance index data based on the Pearson correlation coefficient to obtain the resource data to be predicted and the performance index data. Correlation relationship between index data; threshold definition module, used to define the correlation threshold based on the correlation relationship; time series data acquisition module, used to take performance index data greater than or equal to the correlation threshold as performance index time series data; data extension The module is used to expand the performance index time series data horizontally to obtain training data and test data; the model training module is used to input the training data into the constructed time series neural network model for training, and obtain the trained time series neural network. Prediction model; the data prediction module is used to input the test data into the time series neural network prediction model for prediction operation to obtain the resource prediction result. Based on the preset time interval, the historical resource data of the container load in the cloud native cluster is collected, and the collected historical resource data is deleted, normalized and other preprocessing to obtain the resource data to be predicted; then, based on the Pearson correlation The coefficients are used to sort the relevancy of the resource data to be predicted and the performance indicator data to obtain the correlation between the resource data to be predicted and the performance indicator data to define the relevancy threshold, and obtain the performance indicator time series data based on the relevancy threshold ; and then perform pruning and information extraction on the performance index time series data based on horizontal data expansion to obtain training data and test data, which can realize the effective information of the data is retained on the basis of reducing the input data; then, based on the training data, add The fully connected layer and the time-series neural network model of the attention mechanism are trained to obtain a time-series neural network prediction model that can be used to capture long-term dependent time-series data information, and then the test data is used as the input of the time-series neural network prediction model to obtain predictions A resource prediction result with a higher accuracy rate; then, the prediction operation process is repeatedly performed, and the obtained real-time prediction data is fed back to the user terminal. It can effectively reduce the computational complexity by reducing the input data of the time series neural network model, thereby reducing the forecasting complexity, and to a certain extent, improving the accuracy and efficiency of cloud-native cluster resource forecasting.

To solve the above technical problems, the embodiments of the present application also provide computer equipment. For details, please refer to FIG. 8 , which is a block diagram of a basic structure of a computer device according to this embodiment.

The computer device 8 includes a memory 81 , a processor 82 , and a network interface 83 that communicate with each other through a system bus. It should be noted that only the computer device 8 with components 81-83 is shown in the figure, but it should be understood that implementation of all shown components is not required, and more or less components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, special-purpose Integrated circuit (Application Specific Integrated Circuit, ASIC), programmable gate array (Field-Programmable Gate Array, FPGA), digital processor (Digital Signal Processor, DSP), embedded equipment, etc.

The computer equipment may be a desktop computer, a notebook computer, a palmtop computer, a cloud server and other computing equipment. The computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touch pad or a voice control device.

The memory 81 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disk, etc. In some embodiments, the memory 81 may be an internal storage unit of the computer device 8 , such as a hard disk or a memory of the computer device 8 . In other embodiments, the memory 81 may also be an external storage device of the computer device 8, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Of course, the memory 81 may also include both the internal storage unit of the computer device 8 and its external storage device. In this embodiment, the memory 81 is generally used to store the operating system and various application software installed on the computer device 8, such as the program code of the cloud native resource dynamic prediction method, and the like. In addition, the memory 81 can also be used to temporarily store various types of data that have been output or will be output.

The processor 82 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 82 is typically used to control the overall operation of the computer device 8 . In this embodiment, the processor 82 is configured to run the program code stored in the memory 81 or process data, for example, run the program code of the cloud native resource dynamic prediction method.

The network interface 83 may include a wireless network interface or a wired network interface, and the network interface 83 is generally used to establish a communication connection between the computer device 8 and other electronic devices.

The present application also provides another embodiment, which is to provide a computer-readable storage medium, where the computer-readable storage medium stores a cloud-native resource dynamic prediction program, and the cloud-native resource dynamic prediction program can be processed by at least one The at least one processor executes the steps of the cloud native resource dynamic prediction method as described above.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.

Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the scope of the patent of the present application. This application may be embodied in many different forms, rather, these embodiments are provided so that a thorough and complete understanding of the disclosure of this application is provided. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements for some of the technical features. . Any equivalent structure made by using the contents of the description and drawings of the present application, which is directly or indirectly used in other related technical fields, is also within the scope of protection of the patent of the present application.

Claims

A method for dynamic prediction of cloud native resources, comprising the following steps:

Receive the dynamic prediction request sent by the user terminal;

In response to the dynamic prediction request, read the local database, and obtain the resource data to be predicted and the performance indicator data of the container load in the cloud native cluster;

Rank the to-be-predicted resource data and the performance index data based on the Pearson correlation coefficient, to obtain a correlation between the to-be-predicted resource data and the performance index data;

define a relevance threshold based on the relevance relationship;

Use the performance index data greater than or equal to the correlation threshold as the performance index time series data;

Perform horizontal data expansion on the performance index time series data to obtain training data and test data;

The training data is input into the constructed time series neural network model for training, and the trained time series neural network prediction model is obtained;

Inputting the test data into the time series neural network prediction model to perform a prediction operation to obtain a resource prediction result.
The method for dynamic prediction of cloud native resources according to claim 1, wherein in the response to the dynamic prediction request, a local database is read to obtain resource data to be predicted and performance indicator data of container loads in the cloud native cluster. Before the step, the method further includes:

Collect historical resource data of container loads in cloud-native clusters based on preset time intervals;

The historical resource data is preprocessed to obtain the resource data to be predicted.
The method for dynamic prediction of cloud native resources according to claim 2, wherein the step of preprocessing the historical resource data to obtain the resource data to be predicted comprises:

Delete invalid or abnormal data in the historical resource data to obtain valid time series data;

The effective time series data is normalized to obtain the resource data to be predicted.
The method for dynamic prediction of cloud native resources according to claim 1, wherein after the step of inputting the test data into the time series neural network prediction model to perform a prediction operation to obtain a resource prediction result, the Methods also include:

Repeatedly executing the prediction operation process to obtain real-time prediction data;

The real-time prediction data is fed back to the user terminal.
The method for dynamic prediction of cloud native resources according to claim 1, characterized in that, in the step of inputting the training data into the constructed time-series neural network model for training to obtain the trained time-series neural network prediction model Before, the method further includes:

A preset fully connected layer and an attention mechanism are added to the basic model architecture of the time-series neural network to obtain the time-series neural network model.
A cloud native resource dynamic prediction device, characterized in that it includes:

a request receiving module for receiving the dynamic prediction request sent by the user terminal;

a request-response module, configured to respond to the dynamic prediction request, read the local database, and obtain resource data to be predicted and performance indicator data of the container load in the cloud native cluster;

a correlation ranking module, configured to sort the to-be-predicted resource data and the performance index data based on the Pearson correlation coefficient, to obtain a correlation between the to-be-predicted resource data and the performance index data;

a threshold definition module for defining a correlation threshold based on the correlation relationship;

a time series data acquisition module, configured to use the performance index data greater than or equal to the correlation threshold as the performance index time series data;

a data expansion module, used for horizontal data expansion of the performance index time series data to obtain training data and test data;

A model training module for inputting the training data into the constructed time-series neural network model for training to obtain a trained time-series neural network prediction model;

A data prediction module, configured to input the test data into the time series neural network prediction model to perform a prediction operation to obtain a resource prediction result.
The cloud native resource dynamic prediction device according to claim 6, wherein the device further comprises:

The data collection module is used to collect historical resource data of container loads in the cloud native cluster based on a preset time interval;

The preprocessing module is used for preprocessing the historical resource data to obtain the resource data to be predicted.
The cloud native resource dynamic prediction device according to claim 7, wherein the preprocessing module comprises:

a data deletion unit for deleting invalid or abnormal data in the historical resource data to obtain valid time series data;

A normalization processing unit, configured to perform normalization processing on the valid time series data to obtain the resource data to be predicted.
A computer device, characterized in that it includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the cloud native method according to any one of claims 1 to 5 when the processor executes the computer program Steps of a resource dynamic prediction method.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the cloud-native cloud-native system according to any one of claims 1 to 5 is implemented. Steps of a resource dynamic prediction method.