WO2018161723A1

WO2018161723A1 - Power load forecasting system based on long short-term memory neural network

Info

Publication number: WO2018161723A1
Application number: PCT/CN2018/072372
Authority: WO
Inventors: 杨延东; 邓力; 李书芳; 张贯京; 葛新科
Original assignee: 深圳市景程信息科技有限公司
Priority date: 2017-03-08
Filing date: 2018-01-12
Publication date: 2018-09-13
Also published as: CN106952181A

Abstract

A power load forecasting system (10) based on a long short-term memory neural (LSTM) network, wherein the LSTM network comprises an input layer, an LSTM network layer, and an output layer. The system comprises: an information receiving module (101) used for transmitting input power load data and region feature factor at a historical moment to the input layer; a modeling module (102) used for training and modeling the power load data and the region feature factor at the historical moment by means of the LSTM network layer, in order to generate a deep neural network load forecasting model; a power forecasting module (103) used for forecasting the power load in a region by means of the deep neural network load forecasting model, and generating a forecasting result of the power load in the region by means of a regressor connected to the LSTM network layer; and a result output module (104) used for outputting the forecasting result of the power load in the region by means of the output layer. By constructing a load forecasting model for multi-task learning on the basis of an LSTM network, power consumption loads in multiple regions can be precisely forecasted, and the forecasting effect is improved.

Description

Electric load forecasting system based on long and short time memory neural network

Technical field

The present invention relates to the field of power load prediction technologies, and in particular, to a power load prediction system based on a long and short time memory neural network.

Background technique

The power load forecasting problem aims to predict the electricity demand of single or multiple transmission lines in the power grid. According to the predicted time span, it can be divided into short-term forecast (minutes to one week), medium-term forecast (one month to one quarter) and long-term. Forecast (more than one year). Due to the prior art conditions, it is difficult to efficiently store electrical energy in a large power storage device. Therefore, reducing the remaining power generation as much as possible under the condition of satisfying the power supply requirement is an effective way to reduce the cost and improve the efficiency of power consumption. Therefore, it is necessary to accurately predict the short-term power supply load in the region by using various prediction methods, and it is necessary to plan and guide the power generation enterprises to effectively produce electric energy. Currently, there are many mainstream methods for power load forecasting, like artificial neural networks (Artificial Neural Network, ANN), Support Vector Machine (Support) Vector Machine, SVM), Gaussian Process Regression (Gaussion Process Regression (GPR), Autoregressive Integrated Moving Average Model (ARIMA), etc. Power loads are associated with many hidden variables, such as lighting, wind, holidays, etc. These variables are generally difficult to obtain or quantify, but it is reasonable to assume that cities in the same region have similar hidden variables. Therefore, the power load data of these neighboring cities is highly correlated, and the use of multi-task learning technology will improve the load forecasting accuracy of these similar areas.

Multi-task learning is a technique to improve generalization ability by jointly learning multiple related tasks at the same time. When some parameters in the model are properly shared among tasks, the load forecasting effect of these related tasks can be improved at the same time. In recent years, with the in-depth development of deep learning theory research, it is a very meaningful work to apply the deep learning theory to the power demand forecasting of power systems. The existing neural network-based prediction methods rarely predict the power load across regions at the same time, and the proposed power load prediction model is not accurate.

technical problem

The main object of the present invention is to provide a power load prediction system based on a long-short-time memory neural network, and construct a load forecasting model for multi-task learning based on a long-short-time memory neural network (LSTM) in the field of deep learning, which can accurately predict simultaneously. The electrical load of multiple adjacent areas.

Technical solution

To achieve the above object, the present invention provides a power load prediction system based on a long and short time memory neural network, which is operated in a computer, the computer comprising an input unit and an output unit, the long and short time memory neural (LSTM) network comprising an input layer The LSTM network layer and the output layer, the power load prediction system includes:

An information receiving module, configured to receive, by the input unit, the electrical load data and the regional characteristic factor of the input historical moment, and transmit the electrical load data and the regional characteristic factor of the historical moment to an input layer of the LSTM network;

a model establishing module, configured to import power load data and regional feature factors of a historical time received by an input layer of the LSTM network into the LSTM network layer, and use the LSTM network layer to perform power load data of the historical time and The regional characteristic factors are trained and modeled to train to generate a deep neural network load prediction model, which is a single-layer multi-task deep neural network model or a two-layer multi-task deep neural network model for power supply load prediction. ;

a power prediction module, configured to predict, by using the deep neural network load prediction model, a power load in a required prediction area, and generate a power load prediction result in the area by using a regression device connected to the LSTM network layer;

And a result output module, configured to output, by the output layer, a power load prediction result in a required prediction area to the output unit.

Preferably, the deep neural network load prediction model is expressed as the following formula:

Where t∈[0,24] is the time of day, in hours; d∈{1,2,...,365,366} is the number of days of the year, in days; c is one day Type; y ₁ is historical power load data containing a historical power demand; u ₁ is a real value vector containing regional feature factors; id represents the area identifier of the power demand.

Preferably, the network is an iterative LSTM an improved neural network, the neural network through iteration of the state vector h ₁ hidden layer state transition function f is applied recursively process the input sequence in the hidden layer of time step t The state vector h _{t is} determined by the current input sequence x _t and the hidden layer state vector h _t-1 at the previous moment, and the hidden layer state vector h _t is expressed by the following formula:

.

Preferably, the LSTM network layer includes an input gate IT, an output gate o _t and a forgetting gate f _t, and a memory unit c _t . At time t, the memory unit c _t records all history information up to the current time t and is subjected to an input gate. The three logic gates of i _t , output gate o _t and forget gate f _t , the output values of the three logic gates are between 0 and 1.

Preferably, the control information of forgetting door f _t LSTM an erasable network layer, the gate input i _t LSTM updating the network layer control information, said output information output gate o _t controlling an internal state.

Preferably, the input sequence of the LSTM network is x=(x ₁ , x ₂ , . . . , x _t ), which is input to the LSTM network layer by the input layer, and the output sequence is y=(y ₁ , y ₂ ,. .., y _t ), output from the LSTM network layer by the output layer, where T is the prediction period, x is the historical input data, y is the predicted power load, and the parameter iterative update mode of the LSTM network layer is as follows (1) -(6):

Where x _t is the input sequence at time t, σ is the sigmoid function, ⊙ is the multiplication between elements, W is the input weight, U is the cyclic weight of the hidden layer state h, and V is the influence weight of the historical information, tanh It is the hyperbolic tangent function of the hidden layer state h.

Preferably, the plurality of related tasks of the single-layer multi-task deep neural network model share an identical LSTM network layer, and the output of the same LSTM network layer at time t is represented as h _t ^(s) , wherein the initialization parameters are uniform Random sample values distributed between [-0.1, 0.1].

Preferably, the two related tasks of the dual-layer multi-task deep neural network model are respectively assigned to one LSTM network layer, and each task uses information about the LSTM network layer of another task, and is controlled by a global gating unit. Information reception of a two-layer multi-task deep neural network model.

Preferably, the output of the LSTM network layer of the two-layer multi-task deep neural network model at time is represented as h _t ^(m) and h _t ⁽ⁿ⁾ , wherein initialization of h _t ^(m) and h _t ⁽ⁿ⁾ The parameters are random sample values evenly distributed between [-0.1, 0.1], (m, n) is a given set of related tasks, and the memory information of the MLTM network layer of the mth task is as shown in the formula:

,

among them,

x _t is the input sequence at time t, σ is expressed as sigmoid function, ⊙ is represented as multiplication between elements, W is input weight, U is the cyclic weight of hidden layer state h, V is the influence weight of historical information, tanh is The hyperbolic tangent function of the hidden layer state h.

Beneficial effect

Compared with the prior art, the power load prediction system based on the long and short time memory neural network of the present invention is based on the long and short time memory neural network in the field of deep learning (Long Short-term Memory Neural Network (LSTM) is used to build a load forecasting model for multi-task learning to further improve the prediction effect. The invention proposes a cross-region power supply load prediction model, which can simultaneously predict the power load of multiple regions, and the prediction effect is more accurate than the existing power load prediction model.

DRAWINGS

1 is a block diagram of a preferred embodiment of a power load prediction system based on a long and short time memory neural network of the present invention;

2 is a schematic diagram of a model structure of an LSTM network;

3 is a schematic diagram of a single-layer multi-task deep neural network model for power supply load prediction;

4 is a schematic diagram of a two-layer multi-tasking deep neural network model for power supply load prediction.

The objectives, features, and advantages of the present invention will be described in conjunction with the embodiments of the invention.

BEST MODE FOR CARRYING OUT THE INVENTION

The specific embodiments, structures, features and functions of the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Referring to Figure 1, Figure 1 is a block diagram of a preferred embodiment of a power load prediction system based on a long and short time memory neural network of the present invention. In the embodiment, the power load prediction system 10 based on the long and short time memory neural network is installed and runs in the computer 1. The computer 1 further includes, but is not limited to, the input unit 11, the storage unit 12, the processing unit 13, and Output unit 14. The input unit 11 is an input device of a computer, such as an input keyboard or a mouse. The storage unit 12 can be a read only memory unit ROM, an electrically erasable storage unit EEPROM, a flash memory unit FLASH, or a solid state hard disk. The processing unit 13 may be a central processing unit (CPU), a microcontroller (MCU), a data processing chip, or an information processing unit having a data processing function. The output unit 14 is an output device of the computer 1, such as a display or a device such as a printer.

The power load prediction system 10 based on the long and short time memory neural network can be based on a long and short time memory neural network in the field of deep learning (Long Short-term Memory Neural Network (hereinafter referred to as LSTM network 2) is used to construct a load forecasting model for multi-task learning to further enhance the effect of regional power load forecasting. Referring to FIG. 2, the LSTM network 2 includes an input layer 21, an LSTM network layer 22, and an output layer 23.

In this embodiment, the power load prediction system 10 based on the long and short time memory neural network includes, but is not limited to, the information receiving module 101, the model establishing module 102, the power prediction module 103, and the result output module 104. The term "module" as used herein refers to a series of computer program instructions that can be executed by the processing unit 13 of the computer 1 and that are capable of performing fixed functions, which are stored in the storage unit 12 of the computer 1.

The information receiving module 101 is configured to receive the input power load data and the regional characteristic factor of the historical moment through the input unit 11, and transmit the power load data and the regional characteristic factor of the historical moment to the input layer of the LSTM network 2 21; Specifically, the typical electric load demand forecasting problem is affected by various regional characteristic factors, including regional time, holidays, weather, and economic indicators, and the historical electric load data refers to required The electric load data information of the historical time in the predicted area. In the present embodiment, the historical power load data and the regional characteristic factors are collected by the user from the required prediction area, and the information receiving module 101 receives the historical power load data and the regional characteristic factors from the input unit 11 and transmits to the LSTM network 2 Input layer 21.

The model establishing module 102 is configured to import the power load data and the regional feature factors of the input layer 21 of the LSTM network 2 into the LSTM network layer 2, and pass the LSTM network layer 2 to the historical moment. The electrical load data and regional characteristic factors are trained and modeled to train the deep neural network load prediction model. In this embodiment, the deep neural network load prediction model is a single-layer multi-task deep neural network model or a two-layer multi-task deep neural network model for power supply load prediction. The model building module 102 uses the historical power load data and the regional characteristic factors to perform load prediction modeling to generate a deep neural network load prediction model, and the deep neural network load prediction model can be expressed as the following formula:

The variables in the above formula are as follows: t∈[0,24], which is the time of day, in hours; d∈{1,2,...,365,366} is the number of days in the year, in days Unit; c is the type of day, such as Monday to Sunday, holidays, etc.; y ₁ is a real value vector containing historical power load data within a historical power demand; u ₁ is a real value vector containing regional feature factors, Such as temperature, economic indicators and other data; id represents the regional identification of electricity demand.

After sampling and collecting the above eigenvectors, the model can be constructed by determining the state transition function in the above equation and then predicting the power load in an area. The invention adopts an iterative neural network (Recurrent Neural Network, RNN) is an improved network long-term memory neural network (LSTM) network for modeling. The structure and principle of the network model will be described in detail below.

An iterative neural network (RNN) is a network that processes arbitrarily long input sequences by recursively applying a state transfer function f to the hidden layer state vector. The hidden layer state vector h _t at time step _{t is} determined by the current input x _t and the hidden layer state vector h _t-1 at the previous moment, as shown in the following equation:

The above formula can be regarded as a dynamic system, and the state of the system changes with time according to a certain law. h _t is the state of the system. In theory, the iterative neural network (RNN) can approximate any dynamic system. Traditionally, the strategy for modeling time series is to map the input sequence to a fixed-length vector using an iterative neural network (RNN) and then input it into the regression, which gives the prediction. However, during the training process, multiple RNNs based on the state transition function will exponentially increase or decrease the gradient vector after inputting the long sequence. This is the problem of gradient disappearance or gradient explosion faced by RNNs. In this case, it is difficult for multiple RNNs to learn the long-term correlation problem of the sequence.

In this embodiment, the Long and Short Time Memory Neural Network (LSTM) network is an improved iterative neural network (RNN) model, which effectively solves the gradient disappearance or explosion faced by a simple iterative neural network by introducing a logic gate mechanism. The problem is that the deep network model can learn the long-term dependence of time series. The key to the LSTM network is the introduction of a set of Memory Units that allow the network to learn when to forget historical information and when to update memory cells with new information.

As shown in FIG. 2, FIG. 2 is a model structure diagram of an LSTM network. In this embodiment, the LSTM network 2 is composed of an input layer 21, an LSTM network layer 22, and an output layer 23, and the structure is as shown in FIG. 2. LSTM the network layer 22 includes an input gate i _t (input gate), output gate o _t (output gate) and the gate forgetting f _t (forget gate) memory cell and a c _t. At time t, the memory unit c _t records all the history information up to the current time and is controlled by three logic gates: input gate i _t (input gate), output gate o _t (output Gate) and forget gate f _t (forget gate). They are capable of simulating input, read and reset operations between nerve cells. The output values of these three logic gates are between 0 and 1.

Assuming that the input sequence of the LSTM network 2 is x=(x ₁ , x ₂ , . . . , x _t ), it is input from the input layer 21 to the LSTM network layer 22, and the output sequence is y=(y ₁ , y ₂ , .. ., y _t ), output from the LSTM network layer 22 by the output layer 23. Where T is the forecast period, x is the historical input data (eg historical load, weather conditions, economic indicators, etc.) and y is the predicted load. To achieve this goal, the parameter iterative update of the LSTM network layer 22 is as shown in the following equations (1)-(6):

Where x _t is the input sequence at time t, σ is the sigmoid function, tanh is the hyperbolic tangent function of the hidden layer state h, ⊙ is represented as the multiplication between elements, W is the input weight, and U is the loop of the hidden layer state. Weight, V is the influence weight of historical information, and these weight parameters are obtained through model training. It can be seen that the forgetting gate f _t controls the information deletion in the LSTM network layer 22; the input gate i _t controls the information update in the LSTM network layer 22; and the output gate o _t controls the information output of the internal state of the LSTM network layer 22.

In this embodiment, the input gate i _t , the output gate o _t , the forgetting gate f _{t ,} and the memory unit c _t enable the LSTM network layer 22 to adaptively select forgetting, memorizing, and outputting memory information. If important information content is detected, The forgetting gate f _t will be turned off, so that the information will be utilized in multiple time steps, which is equivalent to capturing a long-term dependency information; on the other hand, when the forgetting gate f _{t is} turned on, the LSTM network layer 22 The reset memory state will be selected.

Most of the existing neural network-based load forecasting methods are single-task learning modes. These methods are limited by the small number of training samples, and cannot fully learn the network structure and parameters. To solve this problem, these models have joined the unsupervised pre-training phase. This unsupervised pre-training method is effective for improving the final performance, but it is not the direct task of optimizing the system. Since the features learned from a task by the deep neural network model can be applied to improve the learning of the remaining related tasks, the deep neural network model is very suitable for multi-task learning. The present invention proposes two deep neural network load prediction models based on multi-task learning architecture, which are a single-layer multi-task deep neural network model for power supply load prediction and a two-layer multi-task deep neural network model for power supply load prediction. The specific model structure is shown in Figures 3 and 4.

Referring to FIG. 3 and FIG. 4, FIG. 3 is a schematic diagram of a single-layer multi-task deep neural network model for power supply load prediction; and FIG. 4 is a schematic diagram of a two-layer multi-task deep neural network model for power supply load prediction. In Figure 3, a plurality of related tasks share an identical LSTM network layer 22, the output of which is represented as h _t ^(s) at time t. In Figure 4, two related tasks are each assigned to an LSTM network layer 22 such that each task can use information about the LSTM network layer 22 of another task. It is worth noting that in Figure 4, given a set of related tasks (m, n), each task has its own LSTM network layer 22, representing the output of the pair of LSTM network layers 22 at time t as h _t ^{( m)} and h _t ⁽ⁿ⁾ , in order to better control the flow of shared information from one task to another, the present invention uses a global gating unit 31 to give the model the ability to determine how much information should be received. Based on the above formula (4), the memory content of the LSTM network layer 22 that redefines the mth task is as shown in the formula (7):

(7)

among them,

The remaining parameter settings are consistent with the standard LSTM network layer 22, ie: x _t is the input sequence at time t, σ is the sigmoid function, W is the input weight, U is the cyclic weight of the hidden layer state h, and V is the historical information. The influence weight, tanh is the hyperbolic tangent function of the hidden layer state h.

The power prediction module 103 is configured to predict a power load in a required prediction area by using the deep neural network load prediction model, and generate a power load prediction result in the area through the regression unit 30. The invention can predict the electric load in the required prediction area by the single-layer multi-task deep neural network model or the double-layer multi-task deep neural network model and generate the electric load prediction result in the area. The two models proposed by the present invention can jointly learn two related tasks at the same time, and the LSTM network layer 22 of the last layer of the model is connected to the regression unit 30, such as a Support Vector Regressor, etc., through the regression device 30. The predicted power load value is output.

In this embodiment, the output of the LSTM network layer in the single-layer multi-tasking deep neural network model at time is expressed as , wherein the initialization parameter is a random sample value uniformly distributed between [-0.1, 0.1]. The neutralization initialization parameter of the LSTM network layer 22 of the two-layer multi-tasking deep neural network model is a random sample value uniformly distributed between [-0.1, 0.1]. The minimum error sum of squares is used as the loss function, and the error back propagation algorithm is used for training. The cross-validation method is used to find the hyperparameter of the model. The error back propagation algorithm and the cross-validation method are all prior art in the prior art, and the present invention does not specifically describe them.

The result output module 104 outputs the power load prediction result in the required prediction area to the output unit 14 through the output layer 23; specifically, the output unit 14 outputs the area generated by the regression unit 30 through the output layer 23. The result of the electric load forecast within the group, that is, the sum of the electric load values of a group of related tasks.

Compared with the prior art, the present invention has the following technical advantages: capable of jointly learning and storing short-term fluctuation information, seasonality and trend information contained in a long-time load sequence, and is suitable for multi-task high-dimensional time series prediction problems. . The power load forecasting system based on the long-short-time memory neural network of the present invention constructs a load forecasting model for multi-task learning based on the long-short-time memory neural network (LSTM) in the depth learning field to further improve the prediction effect. The invention proposes a cross-region power supply load prediction model, which can simultaneously predict the power load of multiple regions, and the prediction effect is more accurate than the existing power load prediction model.

The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and equivalent structural or equivalent functional changes made by the description of the present invention and the accompanying drawings may be directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Industrial applicability

Claims

An electric load forecasting system based on a long-and-short-time memory neural network running in a computer, the computer comprising an input unit and an output unit, wherein the long-term memory neural network (LSTM) network comprises an input layer, an LSTM network layer, and An output layer, the power load prediction system includes: an information receiving module, configured to receive, by the input unit, the power load data and the regional characteristic factor of the input historical moment, and transmit the power load data and the regional characteristic factor of the historical moment to An input layer of the LSTM network; a model establishing module, configured to import power load data and regional feature factors of a historical time received by an input layer of the LSTM network into the LSTM network layer, and pass the LSTM network layer The power load data and regional characteristic factors of the historical moment are trained and modeled to train a deep neural network load forecasting model, which is a single-layer multi-task deep neural network model for power supply load prediction or Two-layer multi-task deep neural network model; power prediction mode And using the deep neural network load prediction model to predict a power load in a required prediction area, and generating a power load prediction result in the area by a regression unit connected to the LSTM network layer; a result output module, And a method for outputting, by the output layer, a power load prediction result in a required prediction area to the output unit.
The power load prediction system based on a long-term and short-term memory neural network according to claim 1, wherein the deep neural network load prediction model represents the following formula:
. Where t∈[0,24] is the time of day, in hours; d∈{1,2,...,365,366} is the number of days of the year, in days; c is one day Type; y 1 is historical power load data containing a historical power demand; u 1 is a real value vector containing regional feature factors; id represents the area identifier of the power demand.
The power load forecasting system based on long-short-time memory neural network according to claim 1, wherein said LSTM network is an improved iterative neural network that recursively returns a hidden layer state vector h t Applying the state transfer function f to process the network of input sequences, the hidden layer state vector h t at time step t is determined by the current input sequence x t and the hidden layer state vector h t-1 at the previous moment, the hidden layer state The vector h t is expressed by the following formula:
.
The power load prediction system based on the long-short-time memory neural network according to claim 1, wherein the LSTM network layer includes an input gate i t , an output gate o t and a forgetting gate f t , and a memory unit c t time t, C t the recording unit to the memory of all the history information until the current time t and I t being an input gate, the output of gate O t F t and three forgetting door control logic gate, the output value of three logic gates are in Between 0 and 1.
The power load prediction system based on the long-short-time memory neural network according to claim 4, wherein the forgetting gate f t controls information deletion of the LSTM network layer, and the input gate i t controls information of the LSTM network layer Update, the output gate o t controls the information output of the internal state.
The power load prediction system based on long-short-time memory neural network according to claim 4, wherein the input sequence of the LSTM network is x=(x 1 , x 2 , . . . , x t ), input The layer is input to the LSTM network layer, and the output sequence is y=(y 1 , y 2 , . . . , y t ), which is output from the LSTM network layer by the output layer, where T is the prediction period and x is the historical input data, y It is predicted the power load, and the parameter iterative update mode of the LSTM network layer is as shown in the following formulas (1)-(6):
(1)
(2)
(3)
(4)
(5)
(6) where x t is the input sequence at time t, σ is the sigmoid function, ⊙ is the multiplication between elements, W is the input weight, U is the cyclic weight of the hidden layer state h, and V is the influence of historical information. Weight, tanh is the hyperbolic tangent function of the hidden layer state h.
The power load forecasting system based on a long-short-time memory neural network according to claim 1, wherein a plurality of related tasks of said single-layer multi-task deep neural network model share an identical LSTM network layer, the same LSTM The output of the network layer at time t is denoted as h t (s) , where the initialization parameters are random sample values evenly distributed between [-0.1, 0.1].
The power load forecasting system based on long-short-time memory neural network according to claim 1, wherein two related tasks of the two-layer multi-task deep neural network model are respectively assigned to an LSTM network layer, and each task is separately used. Another task is related to the LSTM network layer and controls the information reception of the two-layer multi-tasking deep neural network model through a global gating unit.
The power load prediction system based on long-term and short-term memory neural network according to claim 8, wherein the output of the LSTM network layer of the two-layer multi-task deep neural network model at time is expressed as h t (m) and h t (n) , where the initialization parameters of h t (m) and h t (n) are random sample values uniformly distributed between [-0.1, 0.1], and (m, n) is a given set of related tasks The memory information of the LSTM network layer of the mth task is as shown in the formula:
,among them,
x t is the input sequence at time t, σ is the sigmoid function, W is the input weight, U is the cyclic weight of the hidden layer state h, V is the influence weight of the historical information, and tanh is the hyperbolic tangent function of the hidden layer state h .