CN114936620A

CN114936620A - Sea surface temperature numerical value forecast deviation correction method based on attention mechanism

Info

Publication number: CN114936620A
Application number: CN202210209290.5A
Authority: CN
Inventors: 汪祥; 朱俊星; 费童涵; 张卫民; 陈祥国; 王辉赞; 陈妍
Original assignee: National University of Defense Technology
Current assignee: National University of Defense Technology
Priority date: 2022-03-04
Filing date: 2022-03-04
Publication date: 2022-08-23
Anticipated expiration: 2042-03-04
Also published as: CN114936620B

Abstract

The invention discloses a sea surface temperature numerical value forecast deviation correction method based on an attention mechanism, which comprises the following basic steps of: preprocessing the forecast data of the mode to be corrected, and constructing an input sequence; constructing a sea surface temperature correction model; correcting the mode data by using the constructed model; and evaluating the correction precision of the model. The method not only considers the influence of the spatial distribution of the training data and adds characteristic factors such as salinity, but also considers the importance of historical information. The model can effectively extract the space-time dependence relationship between the sea temperature field data, thereby realizing the SST correction with high precision.

Description

Sea surface temperature numerical value forecast deviation correction method based on attention mechanism

Technical Field

The invention belongs to the technical field of ocean weather forecast correction, and particularly relates to a sea surface temperature numerical value forecast deviation correction method based on an attention mechanism.

Background

The oceans occupy over 70% of the surface area of the earth, and are closely related to human activities. Sea surface temperature is an important physical quantity in global climate research, marine ecosystem research and marine related applications. The change of the seawater temperature has very important influence on activities such as navigation, marine disaster prevention and reduction, marine fishery and the like, so that the accurate observation and forecast of the sea surface temperature have important significance. The numerical model prediction is a common prediction method in ocean prediction, but the numerical prediction mode cannot completely describe various physical processes in the ocean, the model also has the problems of uncertainty of an initial field, calculation errors which are difficult to avoid in the model numerical solving process and the like, and the prediction result still has certain errors, so that the prediction result of a numerical prediction product needs to be further corrected. The machine learning based predictive model is a pure data-driven predictive model, although it has great advantages in capturing the non-linear relationship between the predictor and the target of the forecast. However, with the progress of science and technology and society, element prediction developed based on a deep learning method is difficult to meet the current business prediction requirement, and the prediction result based on a numerical pattern has an error which is difficult to avoid. Neither the data-driven machine learning method nor the theoretical-driven numerical model method can meet the demand of marine factor prediction accuracy. Therefore, it is urgently needed to forecast marine elements by fusing a theoretical-driven numerical pattern method and a data-driven deep learning method. The machine learning method is utilized to carry out post-processing and other work on the numerical mode prediction result to correct the prediction error of the numerical prediction product, so that the accuracy of prediction of various elements is improved. In terms of how the two are fused, it is imperative to develop a deep learning method to perform post-processing of the model result and other work to correct the prediction error of the numerical prediction product, so as to improve the accuracy of the prediction of various factors.

Disclosure of Invention

In view of this, the invention provides a sea surface temperature numerical prediction deviation correction method based on an attention mechanism, and the sea surface temperature correction method with the attention mechanism based on ConvLSTM and CBAM, which comprises the steps of data preprocessing, input sequence construction, correction model construction, correction of model data, correction precision evaluation and the like.

The invention discloses a sea surface temperature numerical value forecast deviation correction method based on an attention mechanism, which comprises the following steps of:

the method comprises the following steps: preprocessing the mode forecast data and the remote sensing satellite sea temperature data, and constructing an input sequence, wherein the input sequence comprises the following steps:

(1) acquiring mode forecast data and remote sensing satellite sea temperature data serving as reference values, and extracting marine environment data;

(2) constructing a time column, acquiring the characteristic quantity of time through a sliding window, and adding ocean characteristic influence factors including the U and V vectors of the sea temperature, the salinity and the water flow to obtain an input history SST sequence of the model;

(3) standardizing the historical SST sequence data obtained in the step;

step two: constructing a correction model, comprising:

(1) spatial feature extraction: performing convolution on a cube formed by superposing a three-dimensional kernel and a plurality of continuous matrixes by utilizing three-dimensional convolution, and extracting the spatial dependence characteristics of the sea temperature data and the relation between a plurality of environment variables;

(2) improving the utilization rate of the spatial characteristics of the three-dimensional convolution network by utilizing a 3D-CBAM attention mechanism for the data sequence after three-dimensional convolution, and displaying the importance of different environment variables to the result; the 3D-CBAM attention mechanism consists of a channel attention module and a space attention module;

multiplying Mc and the input feature matrix F to obtain an input matrix Ms of the space attention module;

multiplying Ms and the input F' of the module to obtain a finally generated characteristic matrix;

(3) inputting the characteristic matrix data sequence obtained in the step into a ConvLSTM part of the model; adding a self-defined Attention layer behind the ConvLSTM network by using an Attention mechanism, and distributing time Attention weight to the hidden layer state of each time step by using the hidden layer state of each step of the ConvLSTM model;

adjusting the final ConvLSTM output to obtain a final correction result;

(4) setting up and training the correction model obtained in the step, continuously adjusting parameters, and preferentially selecting the parameters to obtain a sea surface temperature correction model;

step three: correcting the mode forecast data by using a correction model;

(1) inputting the time column forecast data after the standardization treatment into the sea surface temperature correction model obtained in the step two to obtain an output result after correction;

(2) performing anti-standardization processing on the model output result, wherein the anti-standardization method corresponds to the standardization method in the first step, and processing to obtain an SST value after correction;

step four: and evaluating the result precision after the model is corrected by using MAE, MAPE, MSE and RMSE evaluation indexes.

Furthermore, a bilinear interpolation method is adopted to unify the space-time resolution of the forecast data and the remote sensing satellite observation data.

Further, for the historical data sequence X, where the SST data at any time T and Xt composed of other marine environment variables are grid data of W × H × C specification, the input of the whole model is a five-dimensional tensor expressed as B × T × C × W × H, where B is the number of a batch of training samples, T is the length of the sequence data, W and H are the length and width of the SST field, and C is the number of the added marine environment variables.

Further, the normalization formula is as follows:

wherein X _max ,X _min Are the maximum and minimum values in the sequence data, respectively.

Further, the channel attention module has the following calculation formula:

Mc(F)＝σ(MLP(MaxPool3D(F))+MLP(AvgPool3D(F)))

where MLP denotes multilayer perceptron, MaxPool3D denotes maximum pooling, AvgPool3D denotes average pooling, F is the input feature matrix;

the input matrix Ms of the spatial attention module is:

Ms(F)＝σ(f ^3×3 ([MaxPool3D(F)；AvgPool3D(F)]))

where f is a convolution operation of 3 x 3.

Further, the calculation formula of the ConvLSTM network is as follows:

i _t ＝σ(w _xi *x _t +w _hi *h _t-1 +w _ci ·c _t-1 +b _i )

f _t ＝σ(w _xf *x _t +w _hf *h _t-1 +w _cf ·c _t-1 +b _f )

c _t ＝f _t ·c _t-1 +i _t ·tanh(w _xc *x _t +w _hc *h _t-1 +b _c )

o _t ＝σ(w _xo *x _t +w _ho *c _t-1 +w _co ·c _t +b _o )

h _t ＝o _t ·tanh(c _t )

wherein i _t Denotes an input gate, f _t Indicating forgetting to leave door o _t Represents an output gate, c _t Representing the cell state, w is a weight matrix, w _xi Is an input gate x _t Weight matrix of w _xf Is a forgetting gate x _t Weight matrix of, w _xc Is x in the calculation of the cell state _t Weight matrix of w _ho Is an output gate x _t B is an offset term, b _i Is an offset term of the input gate, b _f Is a biased term of a forgetting gate, b _c Is a bias term for the cell state, b _o Is the offset term of the output gate is x _t Is an input matrix, h _t-1 Is the hidden layer state at time t-1, c _t-1 Is the memory state at time t-1.

Further, note that the layer will ConvLSTM output h per iteration _t As an input; then obtaining the weight AT (h) of each output vector through Softmax operation _t ) (ii) a Finally, attention weight AT (h) _t ) And hidden layer state h _t Multiplying to obtain final correction result Y _t The calculation formula is as follows:

wherein, W is the weight matrix.

Further, a mean square error is adopted as a loss function, and a calculation formula of the loss function is as follows:

where n represents the number of grid points in the grid point data,

is the true value data of the lattice, y _i Is the data of the network correction; after the model is built, setting the input dimension of the model and the time step length of input data; setting a model optimizer and a learning rate; setting the number of the cryptomelanic ganglion points; setting the iteration times of the model; and continuously adjusting parameters, checking the convergence degree of the model according to the model loss, and preferentially selecting the convergence degree parameter to form a final sea temperature correction model.

Further, the mean square error MSE, the mean square error RMSE, the mean absolute error MAE, and the mean absolute percentage error MAPE are calculated as follows:

wherein y is _i In order to be the true value of the value,

for the estimation, n is the number of samples.

The invention has the following beneficial effects:

the change rule of sea surface temperature data is mined based on the time-space sequence, so that a good correction effect can be obtained, and the correction precision is high;

the CBAM module of the invention is very small, thus reducing the training time of the model, having less parameter quantity, faster training speed and good performance.

Drawings

FIG. 1 is a flow chart of the present invention;

FIG. 2 is a diagram of a correction model framework;

FIG. 3 is a diagram of the correction effect of the correction model at different time step lengths;

FIG. 4 is a diagram of the correction effect of the correction model at different learning rates;

fig. 5 is a diagram of the correction effect of the correction model under different training times.

Detailed Description

The invention is further described with reference to the accompanying drawings, but the invention is not limited in any way, and any alterations or substitutions based on the teaching of the invention are within the scope of the invention.

The invention relates to a sea surface temperature correction method with an attention mechanism based on ConvLSTM and CBAM, which comprises the steps of data preprocessing, input sequence construction, correction model construction, correction of mode data, correction precision evaluation and the like. The specific flow chart is shown in fig. 1.

The first step is as follows:

1. and preprocessing the mode forecast data and the remote sensing satellite sea temperature data used as the label value to construct an input sequence. By adopting a bilinear interpolation method, the space-time resolution of the unified forecast data and the remote sensing satellite observation data is 0.25 degrees multiplied by 0.25 degrees every day.

2. For the historical data sequence X, the SST data at any time T and Xt composed of other marine environment variables are grid data of W × H × C specification, so the input of the whole model is a five-dimensional tensor expressed as B × T × C × W × H. Where B is the number of training samples in a batch and T is the length of the sequence data. W and H are the length and width of the SST field, and C is the number of added marine environmental variables. In the experiment, the length H and width W are longitude and latitude. The characteristic amount of time is obtained by a sliding window, for example, if the day SST is corrected with historical data of the past 3 days, the length of the time series is 4, i.e., the value of T. In the experiment, three influencing factors of salinity, U vector and V vector of water flow are added besides sea temperature, so that C is 4 in the experiment. And finally, obtaining a 5-dimensional tensor which is used as an input sequence of the model. U is the east direction and V is the north direction. Sometimes, U refers to the weft velocity and V refers to the warp velocity.

3. And normalizing the obtained input data sequence, and taking the normalized data sequence as input data of the CBAM-ConvLSTM model, wherein the input dimension is B multiplied by T multiplied by C multiplied by W multiplied by H.

The second step comprises the following concrete steps:

and establishing a correction model CBAM-ConvLSTM. The framework of the CBAM-ConvLSTM method is shown in FIG. 2.

1. And (5) extracting spatial features. In this step, a three-dimensional convolution is used to extract the association between the spatially dependent features of the sea temperature data and the various environmental variables. Three-dimensional convolution is developed on the basis of two-dimensional convolution. The three-dimensional convolution is a convolution of a three-dimensional kernel with a cube formed by superimposing a plurality of continuous matrices.

2. And 3D-CBAM attention mechanism is utilized to improve the utilization rate of the space characteristics of the three-dimensional convolution network for the data sequence after three-dimensional convolution, and the importance of different environment variables to the result is displayed. The input feature matrix F (BXTXH multiplied by C) is respectively subjected to global max pooling and global average pooling based on width and height to obtain two BXT multiplied by 1 multiplied by C feature maps, and then the two feature maps are respectively sent into a two-layer neural network (MLP), wherein the first layer of neuron number is C/r (r is a reduction rate), the activation function is Relu, the second layer of neuron number is C, and the two-layer neural network is shared. And then, performing summation operation on the characteristics output by the MLP, and performing sigmoid activation operation to generate a final channel attention matrix, namely Mc. Finally, the Mc and the input feature matrix F are multiplied to generate the input feature F' required by the space attention module. And taking the feature matrix F' output by the channel attention module as an input feature matrix of the module. Firstly, making a channel-based global max and global average firing to obtain two B × T × H × W × 1 feature matrixes, and then making a channel splicing operation (concat) on the basis of the channel for the 2 feature graphs. Then, after a convolution operation, the dimensionality reduction is 1 channel. And generating a spatial attention matrix (Ms) through a sigmoid activation function. And finally multiplying the Ms and the input F' of the module to obtain a finally generated feature matrix X.

3. Inputting X obtained in the above steps into ConvLSTM part of the model, and setting the sea temperature at a certain time, wherein the interval time is one day, so that the hidden layer state at the previous time is h _t-1 The last memory state is c _t-1 . ConvLSTM includes a forgetting gate f _t And input gate i _t And an output gate o _t . At the current time t, forget the door f _t Responsible for controlling the last moment c _t-1 How much to save to the current moment c _t (ii) a Input gate i _t Is responsible for controlling how much the instant state at the current moment is input into the current unit state c _t (ii) a Output gate o _t Is responsible for controlling the current cell state o _t How many hidden layer outputs h as the current time _t . The calculation formulas are respectively as follows:

i _t ＝σ(w _xi *x _t +w _hi *H _t-1 +w _ci ·C _t-1 +b _i )

f _t ＝σ(w _xf *x _t +w _hf *H _t-1 +w _cf ·C _t-1 +b _f )

o _t ＝σ(w _xo *x _t +w _ho *H _t-1 +w _co ·C _t +b _o )

wherein, w _xi 、w _xf 、w _xo Weight matrices for input gate, forgetting gate and output gate, respectively, b _i 、b _f 、b _o Are respectively the offset terms of an input gate, a forgetting gate and an output gate, and sigma is a sigmoid function.

Current cell state c _t By forgetting door f _t Last time cell state c _t-1 And input gate i _t The current cell state is determined together with the current input cell state, and the calculation formula is as follows:

C _t ＝f _t ·C _t-1 +i _t ·tanh(w _xc *x _t +w _hc *H _t-1 +b _c )

wherein, w _xc Is a weight matrix of the input cell states, b _c Is a bias term for the input cell state, and tanh is the hyperbolic tangent function.

Hidden layer output value h of ConvLSTM at current moment _t From an output gate o _t And the current cell state c _t Jointly determining, the calculation formula is as follows:

H _t ＝o _t ·tanh(C _t )

meanwhile, an Attention mechanism is used in the stage, a user-defined Attention layer is added behind the ConvLSTM network, the hidden layer state of each step of the ConvLSTM model is fully utilized, and time Attention weight is distributed to the hidden layer state of each time step. Note that the layer will output h for each iteration of ConvLSTM _t As an input; then obtaining the weight AT (h) of each output vector through Softmax operation _t ) (ii) a Finally, attention weight AT (h) _t ) And hidden layer state h _t Multiplying to obtain the final correction result Y _t The calculation formula is as follows:

wherein, W is a weight matrix.

The entire SST prediction correction model can be expressed as:

the Mean Square Error (MSE) is used as a LOSS function (LOSS), and the calculation formula is as follows:

where n represents the number of grid points in the grid point data,

is the true value data of a lattice point, y _i Is the data of the network subscription. Preferably, after the model is built, setting the input dimension of the model and the time step length of input data; setting a model optimizer and a learning rate;setting the number of the cryptomelanic ganglion points; setting the iteration times of the model; and continuously adjusting parameters, checking the convergence degree of the model according to the model loss, and preferentially selecting high-convergence parameters to form a final sea temperature correction model.

Step three: correcting the mode forecast data by using a correction model;

1. inputting the time column forecast data after the standardization processing into the sea surface temperature correction model obtained in the second step to obtain an output result after correction; the historical data is mainly used for training the model and is data of a time sequence, and the data of the current day is used for correcting the data and is data of the current moment. The current day data is typically included in the historical data, but may be corrected when not.

2. Performing anti-standardization processing on the output result of the model, wherein the anti-standardization method corresponds to the standardization method in the step one, and the SST value after correction is obtained through processing, the precision is 0.25 degrees multiplied by 0.25 degrees, and every day;

step four: in order to verify the effectiveness of the CBAM-ConvLSTM model, the model is evaluated by four indexes, namely mean square error MSE, root mean square error RMSE, mean absolute error MAE and mean absolute percentage error MAPE, wherein the MAE reflects the total error of estimation, the RMSE reflects the estimation sensitivity and extreme value effect of sample data, and the smaller the two values, the better the effect is. The calculation method of each index is as follows:

wherein, y _i Which represents the true observed value of the image,

represents the average of the true observations,

the predicted value is represented.

The effect of the 3DCBAM-ConvLSTM based sea surface temperature correction method will be exemplified below. In the experiment, 1/12.5-degree global mixed coordinate marine mode HYCOM forecast data issued by the National Oceanic and Atmospheric Administration (NOAA) is used as forecast data to be corrected, and NOAA 1/4-degree Daily OI SST Analysis satellite remote sensing observation data is used as a true value to evaluate correction precision. The experiment was conducted in the sea area of 0.25 ° x 0.25 ° at 110 ° E-114 ° E, 8 ° N-12 ° N latitude, which is in the south sea. Extracting historical SST data in the range from 2019-01-01 to 2019-12-25 for 360 days.

The HYCOM is forecast three hours by three hours, the daily average temperature of the HYCOM is calculated, a bilinear interpolation method is adopted, and the HYCOM forecast data is interpolated to grid points of OISST Analysis to unify the space-time resolution of the forecast data and the remote sensing satellite observation data. Since the numerical difference of the data is obvious, normalization is required. The normalization operation can improve the convergence rate of the model on one hand, improve the accuracy of the model on the other hand, and also prevent gradient explosion of the model. And generating a corresponding time sequence according to the data time, dividing all the processed SST data, taking 75% of the data as a training set for training parameters of a CBAM-ConvLSTM forecasting model, and taking the remaining 25% of the data as a verification set for verifying the learning effect of the model.

The model is constructed by a pytorech, the shapes of the training data and the input data are adjusted, and converted to the required Tensor (Tensor) format in the pytorech framework. And then defining parameters of the CBAM-ConvLSTM model, including input step length, namely input sequence length, hidden layer number, output sequence length and neuron number of each layer. In the experiment, the convolution part of the model comprises a Conv3D layer and a BN layer, wherein the BN layer mainly has the functions of enabling the distribution of input data of each layer in the network to be relatively stable, accelerating the learning speed of the model, relieving the problem of gradient disappearance and having a certain regularization effect. The convolution kernel size in the Conv3D layer is 3 × 3 × 3 when the network is set. The convolution kernel size in convolution attention in the CBAM portion of the model is 3 × 3 × 3. In the ConvLSTM part of the model, only a single ConvLSTM layer is selected because the sequence selected by the experiment is short, the number of hidden layer neurons is 32, and the number of neurons in the output layer is 1. After model parameters are defined, a loss function and an optimizer are defined, an MSE loss function and an Adam optimizer are selected, and then the model is trained by selecting proper training times. And after the model training is finished, inputting test data into the model for testing, and after the output result of the model is subjected to inverse normalization, obtaining the deviation correction value of the sea temperature. The correction effect is checked by comparing the indexes before and after the correction of the deviation.

To demonstrate the effectiveness of the CBAM-ConvLSTM hybrid model presented herein, experimental results were compared to two conventional machine learning algorithms in a sea surface temperature mapping. They are linear regression and SVM support vector machines, respectively. The linear regression analysis model has the advantages of strong anti-interference capability, high training speed, incapability of simulating a nonlinear relation, low accuracy and easiness in under-fitting. The generalization performance of the SVM model is good, overfitting is not easy to occur, and good performance can be obtained under less data. However, SVM is sensitive to missing data, parameters, and kernel functions, and in order to match the input forms of these algorithms, SST and ocean variables are generally used as independent features, so that the spatiotemporal relationship between the variables cannot be considered. The process for realizing the two algorithms comprises the steps of firstly unfolding all samples into a form which can be processed by the algorithms, and calling a sklern machine learning algorithm packet to perform prediction analysis. Furthermore, we compared a series of models and compared them with the CBAM-ConvLSTM model, because there were fewer methods previously used to map two-dimensional sea temperature. This includes an LSTM model that only considers temporal relationships and not spatial relationships, an improved method that incorporates convolution, ConvLSTM models that add only 3dCnn and ConvLSTM models that add only temporal attention, and CONVLSTM hybrid models that add both CNN and AT. Here, we set the experimental parameters learning rate LR to 0.01, the number of iterations Epoch to 300, and SST correction is performed using 3 days of historical data. We will discuss the parameter settings in detail later. Model evaluation used RMSE, MSE, MAE and MAPE. The results of the prediction experiments for different correction methods are shown in table 1.

TABLE 1 comparison of correction results of correction methods

From the comparison in table 1, it can be found that when the traditional machine learning method is used for booking, the accuracy of SVR is higher than that of linear regression. Among other deep learning models, the 3DCNN-CONVLSTM-AT model has the best result. However, the CBAM-CONVLSTM model provided by the inventor can achieve the MSE value of 0.3520, the MAE value of 0.2641 and the MAPE value of 0.9546% in a correction experiment, and the effect is better than that of the other models.

As can be seen from Table 1, the effect of LSTM is superior to correction using traditional machine learning methods, which illustrates the importance of temporal correlation to SST correction. The result of the conventional LSTM improved method ConvLSTM is superior to that of LSTM, and the importance of spatial correlation on SST correction is verified. Meanwhile, the experimental result of ConvLSTM-AT added with the attention mechanism shows that the model is better in performance because different influences of historical SST on the SST to be predicted are taken into consideration and are expressed in the form of distributing different weights. The results of the 3DCNN-ConvLSTM-AT and ConvLSTM-AT predictions were compared, where ConvLSTM-AT was free of convolutional layers and 3DCNN-ConvLSTM-AT was convolutional layers. Under the condition of the same parameters and the same data set, the 3DCNN-ConvLSTM-AT correction precision is obviously higher than that of the ConvLSTM-AT. Experimental results show that the added convolution layer has a certain effect of improving the SST prediction precision, and the main reason for the situation is that the local features extracted by the convolution operation of the ConvLSTM are not obvious enough for SST data. And a convolution layer is added before the model, so that the feature extraction capability of the model is improved, the spatial features of the data in the ConvLSTM model are more obviously represented, and the SST prediction precision is favorably improved.

And the CBAM-Convlstm model adds a CBAM attention mechanism on the basis of a 3DCNN-Convlstm-AT model, the RMSE index is 0.35, and the correction effect is optimal. The CBAM-ConvLSTM model further extracts spatial features and adds weights to the environmental information and the spatial features, so that the information utilization rate is improved, the model is closer to reality, the contained information is more comprehensive, and the SST prediction precision is finally improved. In summary, compared with the conventional machine learning correction methods such as LR and SVR, and the deep learning methods LSTM, ConvLSTM, and ConvLSTM-AT, CBAM-ConvLSTM has the best performance in correcting SST, and the validity of the methods is verified.

The time step is an important parameter of the process of learning the timing information by the model, and the time step herein refers to the time before the input data when the model is trained, for example, when time is 1, that is, the SST is corrected by the historical data of the previous day. Generally, the longer the time step, the more information is available for correction, but the accumulation of errors increases. Therefore, the SST is corrected by respectively taking 1, 3, 5,7,10 and 15 as the timekeeper, and the appropriate timekeeper is determined for correcting the SST by experimentally comparing indexes such as RMSE, MAPE and the like. FIG. 3 shows a comparison of evaluation indexes of CBAM-ConvLSTM corrected SST at different time steps. the timestamp symbolizes information in the time dimension, and the value has an influence on the performance of the model. When in fixation, RMSE indexes under different timekeeper are compared, when the timekeeper is 3, the fixation effect is better than the evaluation indexes when the timekeeper is 1,5,7,10 and 15, and when the timekeeper is more than 10, the result tends to be stable, and the influence of time information on the fixation result is reduced. It follows that in predicting SST, the amount of information in the spatial and temporal dimensions should be modest, and too much or too little will affect the performance of the model, so the above experiments need to be performed. In conclusion, the SST is corrected by taking time to 3.

To determine the optimal number of training sessions (epochs) for a data set, experiments were performed with different epochs. The RMSE reached steady state at 300 points as shown in fig. 4. Therefore, in our experiments, 300 training times were employed, taking into account the accuracy and performance of the model.

The Learning rate (Learning rate) is an important hyper-parameter that determines whether and when the objective function converges to a local minimum. An appropriate learning rate enables the objective function to converge to a local minimum in an appropriate time. It has been found that at the beginning of the training process, when an adaptive optimization algorithm, such as the "Adam" algorithm, is used, the model is in a non-convergent state. Then adjust lr and other hyper-parameters within the stent frame. The first step is from 0.1 to 0.001, the reduction being 10. Then, when the learning rate is 10 ^-2 At level, the model's training and validation loss will be in a steady decline. The learning rate was adjusted and the experiment was carried out, and the result of the experiment is shown in fig. 5. The graph shows that indexes such as RMSE/MAPE change along with the change of lr. The optimum learning rate is 0.01 in terms of Root Mean Square Error (RMSE). According to MAPE, the optimal learning rate is 0.004. Thus, the best learning rate in our dataset is between 10-2 and 10-3.

Complexity and training time analysis, the experimental environment is Windows10, Intel Core i 511, 2.4GHz, 16G RAM, and the algorithm implementation uses python 3.

TABLE 2 number of network parameters and training time for each model

	Parameters	Train(s)	Test(s)
				LSTM	13601	271	1
CONVLSTM	44993	236	1
				CONVLSTM-AT	46079	437	1
3DCNN-CONVLSTM-AT	13197	272	1
				CBAM-CONVLSTM	13560	223	1

Table 2 lists the training times and the number of model parameters for the models used in the experiments. It was found that the CBAM-CONVLSTM model had training parameters approximately 10 times less than ConvLSTM, which made training faster and more practical. The parameter quantity of the 3DCNN-CONVLSTM-AT model is close to that of the CBAM-CONVLSTM model, which shows that the CBAM module is small and the model training time is reduced. The CBAM-CONVLSTM model provided by the invention consumes the least time and has fewer parameters, and the model has good performance.

The invention has the following beneficial effects:

The word "preferred" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as advantageous over other aspects or designs. Rather, use of the word "preferred" is intended to present concepts in a concrete fashion. The term "or" as used in this application is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise or clear from context, "X employs A or B" is intended to include either of the permutations as a matter of course. That is, if X employs A; x is B; or X employs both A and B, then "X employs A or B" is satisfied in any of the foregoing examples.

Also, although the disclosure has been shown and described with respect to one or an implementation, equivalent alterations and modifications will occur to others skilled in the art based upon a reading and understanding of this specification and the annexed drawings. The present disclosure includes all such modifications and alterations, and is limited only by the scope of the appended claims. In particular regard to the various functions performed by the above described components (e.g., elements, etc.), the terms used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (e.g., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein illustrated exemplary implementations of the disclosure. In addition, while a particular feature of the disclosure may have been disclosed with respect to only one of several implementations, such feature may be combined with one or other features of the other implementations as may be desired and advantageous for a given or particular application. Furthermore, to the extent that the terms "includes," has, "" contains, "or variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term" comprising.

Each functional unit in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or a plurality of or more than one unit are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium. The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Each apparatus or system described above may execute the storage method in the corresponding method embodiment.

In summary, the above-mentioned embodiment is an implementation manner of the present invention, but the implementation manner of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be regarded as equivalent replacements within the protection scope of the present invention.

Claims

1. The sea surface temperature numerical value prediction deviation correction method based on the attention mechanism is characterized by comprising the following steps of:

(3) standardizing the historical SST sequence data obtained in the step;

step two: constructing a correction model, comprising:

(1) spatial feature extraction: performing convolution on a cube formed by superposing a three-dimensional kernel and a plurality of continuous matrixes by utilizing three-dimensional convolution, and extracting the spatial dependency characteristics of the sea temperature data and the relation between a plurality of environment variables;

adjusting the final ConvLSTM output to obtain a final correction result;

step three: correcting the mode forecast data by using a correction model;

2. The method for correcting sea surface temperature numerical prediction bias based on the attention mechanism as claimed in claim 1, wherein the temporal and spatial resolution of the forecast data and the remote sensing satellite observation data are unified by adopting a bilinear interpolation method.

3. The method for correcting sea surface temperature numerical prediction deviation based on the attention mechanism is characterized in that for a historical data sequence X, SST data at any time T and Xt consisting of other marine environment variables are grid data of a W X H X C specification, the input of the whole model is a five-dimensional tensor expressed as B X T X C X W X H, wherein B is the number of a batch of training samples, T is the length of sequence data, W and H are the length and the width of an SST field, and C is the number of added marine environment variables.

4. The method of claim 1, wherein the normalization formula is as follows:

5. The method for correcting sea surface temperature numerical prediction deviation according to claim 1, wherein the channel attention module is calculated according to the following formula:

Mc(F)＝σ(MLP(MaxPool3D(F))+MLP(AvgPool3D(F)))

the input matrix Ms of the spatial attention module is:

Ms(F)＝σ(f ^3×3 ([MaxPool3D(F)；AvgPool3D(F)]))

where f is a convolution operation of 3 x 3.

6. The method of correcting for sea surface temperature numerical prediction bias based on attention mechanism of claim 1, wherein the ConvLSTM network is calculated as follows:

i _t ＝σ(w _xi *x _t +w _hi *h _t-1 +w _ci ·c _t-1 +b _i )

f _t ＝σ(w _xf *x _t +w _hf *h _t-1 +w _cf ·c _t-1 +b _f )

c _t ＝f _t ·c _t-1 +i _t ·tanh(w _xc *x _t +w _hc *h _t-1 +b _c )

o _t ＝σ(w _xo *x _t +w _ho *c _t-1 +w _co ·c _t +b _o )

h _t ＝o _t ·tanh(c _t )

wherein i _t Indicating an input gate, f _t Indicating forgetting gate o _t Represents an output gate, c _t Representing the cell state, w is a weight matrix, w _xi Is an input gate x _t Weight matrix of w _xf Is a forgetting gate x _t Weight matrix of w _xc Is x in the calculation of the cell state _t Weight matrix of w _ho Is an output gate x _t B is an offset term, b _i Is an offset term of the input gate, b _f Is a biased term of a forgetting gate, b _c Is a bias term for the cell state, b _o Is the offset term of the output gate is x _t Is an input matrix, h _t-1 Is the hidden layer state at time t-1, c _t-1 Is the memory state at time t-1.

7. The method of claim 1, wherein the attention layer outputs h of ConvLSTM per iteration _t As an input; then obtaining the weight AT (h) of each output vector through Softmax operation _t ) (ii) a Finally, attention is weighted AT (h) _t ) And hidden layer state h _t Multiplying to obtain final correction result Y _t The calculation formula is as follows:

wherein, W is the weight matrix.

8. The method for correcting sea surface temperature numerical prediction deviation based on the attention mechanism as claimed in claim 1, wherein a mean square error is adopted as a loss function, and the calculation formula of the loss function is as follows:

where n represents the number of grid points in the grid point data,

9. The method of claim 8, wherein the mean square error MSE, the root mean square error RMSE, the mean absolute error MAE, and the mean absolute percentage error MAPE are calculated as follows:

wherein y is _i In order to be the true value of the value,

for the estimation, n is the number of samples.