CN115503559B - Fuel cell automobile learning type cooperative energy management method considering air conditioning system - Google Patents

Abstract

The invention relates to a fuel cell automobile learning type cooperative energy management method considering an air conditioning system, and belongs to the field of new energy automobiles. The method comprises the following steps: s1: acquiring vehicle state parameter information, fuel cell parameter information, power cell parameter information and air conditioning system parameter information of a fuel cell automobile; s2: establishing a fuel cell automobile cooperative energy management model; s3: the method comprises the steps of establishing a fuel cell automobile collaborative energy management optimization control strategy considering an air conditioning system, solving a multi-objective optimization problem comprising hydrogen economy and cabin temperature comfort by combining an SAC algorithm, and controlling the change of refrigerating/heating capacity of an air conditioner to maintain the cabin temperature in a comfort zone while performing energy flow optimization control. The invention can effectively solve the problem of compromise between hydrogen energy consumption and cabin temperature comfort, and optimize the hydrogen economy and cabin temperature comfort of the fuel cell automobile.

Description

Learning-based collaborative energy management method for fuel cell vehicles considering air conditioning system

Technical Field

The present invention belongs to the field of new energy vehicles and relates to a learning-based collaborative energy management method for a fuel cell vehicle taking into account an air-conditioning system.

Background Art

Faced with increasingly severe ecological environmental pollution and fossil fuel shortages, major automobile manufacturers are scrambling to develop new energy vehicles. With the development of fuel cell technology, fuel cell vehicles give full play to the advantages of zero emissions, low energy consumption, and strong endurance, and are considered to be one of the important research directions for achieving sustainable development of future automobiles. Energy management strategy is the core control technology of the multi-power source system of fuel cell vehicles, and its performance directly determines the economic performance of the entire vehicle. In current research, energy management methods are mainly divided into three types: rule-based, optimization-based, and learning-based energy management strategies. However, rule-based and optimization-based energy management methods face the dilemma of being unable to simultaneously meet real-time and optimality; for traditional deep reinforcement learning algorithms, although they can simultaneously achieve real-time and optimality of energy flow optimization, they have certain deficiencies in training data and hyperparameter settings. To this end, the soft-constrained actor-critic algorithm provides a method to solve the above problems.

On the other hand, the air conditioning system, as an indispensable auxiliary equipment for fuel cell vehicles, helps to provide a comfortable riding environment for the passengers in the car. However, the use of the air conditioning system will inevitably increase the energy consumption of fuel cell vehicles, thus affecting the economic performance of the whole vehicle. In the current research on energy management methods for fuel cell vehicles, the energy consumption of the air conditioning system is usually regarded as a constant or ignored. However, due to changes in the driving environment, the heat exchange amount inside and outside the cockpit will change accordingly, which will also cause the power used by the air conditioning system to change.

Therefore, a new fuel cell vehicle energy management method is urgently needed to coordinate the control of the air-conditioning system and the power source components, and to optimize the energy flow in the vehicle while taking into account the changes in the energy consumption of the air-conditioning system.

Summary of the invention

In view of this, the purpose of the present invention is to provide a learning-based collaborative energy management method for fuel cell vehicles taking into account the air-conditioning system. By using a soft actor critic (SAC) algorithm to coordinate the control of the air-conditioning system and power source components of the fuel cell vehicle, the energy flow of the whole vehicle is optimized while ensuring the comfort of the cabin, so as to reduce the energy consumption of the whole fuel cell vehicle.

In order to achieve the above object, the present invention provides the following technical solutions:

A learning-based collaborative energy management method for a fuel cell vehicle considering an air conditioning system comprises the following steps:

S1: Obtain vehicle status parameter information, fuel cell parameter information, power battery parameter information and air conditioning system parameter information of the fuel cell vehicle;

S2: Establish a fuel cell vehicle collaborative energy management model, including: vehicle longitudinal dynamics model, fuel cell model, power battery model, motor model, air conditioning system model and cabin heat load model;

S3: Establish a fuel cell vehicle collaborative energy management optimization control strategy considering the air-conditioning system, and combine the SAC algorithm to solve the multi-objective optimization problem including hydrogen fuel economy and cabin temperature comfort. While performing energy flow optimization control, control the change of air-conditioning cooling/heating capacity to maintain the cabin temperature in a comfortable range; the SAC algorithm is a soft-constrained actor-critic algorithm.

Further, in step S1, the vehicle status parameter information includes: vehicle speed, cabin heat load parameters, motor operating efficiency and transmission system characteristic parameters; the fuel cell parameter information includes: fuel cell power, efficiency and hydrogen energy consumption; the power battery parameter information includes: power battery state of charge, internal resistance and open circuit voltage; the air-conditioning system parameter information includes: air-conditioning system cooling capacity/heating capacity and corresponding power.

Further, in step S2, the longitudinal dynamics model of the whole vehicle is established as:

P _drive =(F _air +F _f +F _i +m ₀ a)·v

_Pdem ＝ _Pb + _Pfc · _ηDC/DC

Among them, _m0 represents the vehicle mass; v represents the vehicle speed; a represents the vehicle acceleration; F _air represents the air resistance; F _f represents the rolling resistance; _Fi represents the acceleration resistance; η _m , η _DC/AC , η _DC/DC and η _motor represent the transmission efficiency, DC/AC converter efficiency, DC/DC converter efficiency and motor efficiency respectively; P _drive , P _dem , P _b and P _fc represent the driving power at the vehicle wheels, the required power, the battery output power and the fuel cell output power respectively.

Further, in step S2, the fuel cell model established is:

η _fc ＝f _η (P _fc )

分别表示为效率和氢能消耗量的拟合函数，可通过插值法计算效率与氢耗。Among them, f _η (·) and

They are respectively expressed as fitting functions of efficiency and hydrogen energy consumption, and the efficiency and hydrogen consumption can be calculated by interpolation method.

Further, in step S2, the power battery model established is:

Among them, I _L represents the power battery current; V _oc represents the power battery open circuit voltage; R _in represents the power battery equivalent internal resistance; SOC ₀ represents the initial SOC; Q _t represents the maximum capacity of the power battery; t ₀ represents the initial time; t _f represents the final time.

Further, in step S2, the motor model established is:

η _m = f _m (ω _m , T _m )

Wherein, ω _m and T _m represent the motor speed and torque respectively; P _m represents the motor output power, and f _m (·) represents the fitting function of the motor working efficiency. The motor working efficiency can be obtained by interpolation method.

Further, in step S2, the air conditioning system model established is:

Wherein, Q _ac represents the cooling capacity or heating capacity of the air-conditioning system; P _ac represents the corresponding power consumption of the air-conditioning system; η _cop represents the performance coefficient of the air-conditioning system.

Further, in step S2, the cabin heat load model established is:

Q _c = ∑ KF (T _out -T _in )

Q _h =145+116n

Q _n ＝m _e ξCp _air (T _out -T _in )

Among them, _Qc , _Qr , _Qh and _Qn represent the heat conduction load, radiation heat load, heat generated by the occupants (according to experience, the heat generated by the driver is about 145W, and each passenger generates about 116W of heat) and ventilation system heat load respectively; K represents the heat transfer coefficient; F represents the heat transfer area of the corresponding shell; _Tout represents the ambient temperature; _Tin represents the cabin air temperature; η represents the permeability; I represents the solar light intensity; _Ai represents the area of the windshield, left and right side windows and rear window; _θi represents the solar incident angle; β represents the shadow factor; n represents the number of passengers in the car; _me represents the air mass passing through the evaporator; ξ represents the air recirculation coefficient; Cp _air represents the indoor air heat capacity; ρ _air and V _air represent the cabin air density and cabin volume respectively.

Further, in step S3, a fuel cell vehicle collaborative energy management optimization control strategy considering the air conditioning system is established, which specifically includes the following steps:

S301: Determine the state space: In order to reflect key environmental information, the power battery SOC, fuel cell output power P _fc , vehicle speed v, and air conditioning system cooling/heating capacity Q _ac are set as state variables to construct the state space S, which can be expressed as:

S＝{SOC, _Pfc ,v, _Qac }

和空调系统制冷/制热容量变化量

设置为动作变量，构建动作空间A，可表示为：S302: Determine the action space: Considering the coordinated energy management of the air conditioning system, not only the power of the power source is allocated, but also the thermal comfort of the cabin temperature should be maintained according to the change of the cooling/heating capacity of the air conditioning system. To this end, the change of the fuel cell output power

and the change in cooling/heating capacity of the air conditioning system

Set as action variable and construct action space A, which can be expressed as:

S303: Establishing a reward function: To ensure the cabin temperature comfort, the cabin temperature is maintained at about 24°C. Therefore, the reward function should also include the optimization item of cabin temperature change. Therefore, the reward function R is set to the weighted sum of the three indicators of hydrogen energy consumption, SOC change and cabin temperature change, expressed as:

R＝-(ζ·fuel(t)+ψ·(SOC(t)-0.7) ² +γ·(T _in -24) ² )

Among them, ζ, Ψ, and γ are weight factors of each optimization item. The trade-off between hydrogen energy consumption and cabin temperature comfort is solved by adjusting the weight factors, thereby solving the multi-objective optimization problem; fuel(t) represents the hydrogen energy consumption at the current moment; SOC(t) represents the state of charge of the power battery at the current moment.

Further, in step S3, the multi-objective optimization problem including hydrogen fuel economy and cabin temperature comfort is solved in combination with the SAC algorithm, which specifically includes the following steps:

S311: Combined with the SAC algorithm to solve the multi-objective optimization problem in energy management, the action entropy value is introduced into the SAC algorithm to make the action output more dispersed, thereby improving the algorithm's exploration ability, ability to learn new tasks and stability. The entropy value is expressed as:

H(π(·|s _t ))=-logπ(·|s _t )

Where H is the entropy of the strategy π(·|s _t ).

S312: During the solution process, the actor network in the agent takes the state s _t as input, outputs the mean and variance of the action Gaussian distribution, and generates the action a _t using the reparameterization technique:

表示函数输出均值和方差；

和

分别表示高斯分布的均值和方差。Where τ _t represents the noise signal sampled from the standard normal distribution;

Represents the function output mean and variance;

and

represent the mean and variance of the Gaussian distribution respectively.

中。S313: After executing action a _t , the vehicle environment feeds back a reward r _t to the agent and transfers to the next state s _t+1 , thus generating the interaction data {s _t , a _t , r _t , s _t+1 } between the environment and the agent and storing it in the experience pool

middle.

S314: Randomly extract a small batch of experience samples from the experience pool. In order to avoid overestimation when maximizing the action state function value and further overestimation when using the own network to calculate the target, introduce an evaluation critic network with parameters θ ₁ , θ ₂ and a target critic network with parameters θ′ ₁ , θ′ _2. Select the target critic network to output a smaller action state function value as the target value. For a specific state s _t and action a _t , the update formula of the soft constraint action value function Q _soft (s _t ,a _t ) in the SAC algorithm is as follows:

Among them, r represents the reward obtained by the vehicle; γ represents the discount factor; α represents the temperature coefficient.

与

之间的均方误差，表示为：S315: When updating the policy network, the evaluation critic network is updated by minimizing the loss function L(θ _i ), which is defined as

and

The mean square error between them is expressed as:

表示为评估评论家网络参数为θ_i时的评价函数，而

表为目标评论家网络参数为θ′_i时的评价函数。in,

It is expressed as the evaluation function for evaluating the critic network parameter θ _i , and

The table shows the evaluation function when the target critic network parameter is θ′ _i .

定义为：S316: The actor network parameter update is achieved by minimizing the KL divergence. The smaller the KL value, the smaller the difference between the rewards corresponding to the output actions, and the better the convergence effect of the strategy; the objective function of the actor network

Defined as:

表示当前时刻下车辆状态s_t、执行动作a_t时的数学期望函数，

表示当前状态为s_t时的策略函数，

表示为策略函数的参数。Where D _KL represents the KL divergence calculation expression; Z(s _t ) is the partition function used to normalize the distribution;

represents the mathematical expectation function of the vehicle state s _t at the current moment and the action a _t ,

represents the policy function when the current state is s _t ,

Represented as parameters of the policy function.

S317: Update the actor network parameters according to the gradient descent method, expressed as:

表示为关于策略函数参数

的下降梯度，

表示为关于当前时刻t下执行动作a_t的下降梯度。in,

Expressed as the policy function parameters

The descent gradient of

It is expressed as the descent gradient of executing action a _t at the current time t.

S318: In the SAC algorithm system, the adjustment of the temperature coefficient α is crucial to the training effect of the SAC algorithm. In different reinforcement learning tasks and training periods, the value of the optimal temperature coefficient is different. In order to achieve automatic adjustment of the temperature coefficient, the optimal temperature coefficient of each step can be updated by solving the minimum value of the objective function in the optimization problem. The objective function is expressed as:

表示为依据策略函数π_t执行动作a_t时的数学期望函数，π_t(a_t|s_t)表示为策略函数，s_t表示为当前时刻t下燃料电池汽车所处的状态，a_t则表示为当前时刻t时依据策略函数执行的动作。Where H ₀ represents the predefined threshold of the minimum policy entropy,

is represented as the mathematical expectation function when action a _t is executed according to the strategy function π _t , π _t (a _t |s _t ) is represented as the strategy function, s _t is represented as the state of the fuel cell vehicle at the current time t, and a _t is represented as the action executed according to the strategy function at the current time t.

The beneficial effects of the present invention are:

1) The present invention designs an energy management strategy based on the soft-constrained actor-critic algorithm, which effectively gets rid of the dependence of traditional deep reinforcement learning algorithms on training data and hyperparameter settings in fuel cell vehicle energy management applications, and is conducive to improving the stability of control tasks in continuous action space.

2) Considering that the changes in energy consumption of the air-conditioning system are usually ignored when designing the energy management problem of fuel cell vehicles, the present invention takes hydrogen energy consumption, SOC maintenance and cabin temperature comfort as optimization goals, builds a collaborative energy management optimization control framework taking the air-conditioning system into account, and realizes the collaborative control of energy management and air-conditioning system.

Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

FIG1 is a flow chart of a fuel cell vehicle collaborative energy management method according to the present invention;

FIG2 is a schematic diagram of the structure of a multi-power source system for a fuel cell vehicle;

FIG3 is a schematic diagram of a cabin heat load model and an air conditioning system structure;

FIG4 is a diagram of a collaborative energy management framework taking into account the air conditioning system and constructed by applying the SAC algorithm in the present invention.

DETAILED DESCRIPTION

The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and the features in the embodiments can be combined with each other without conflict.

Please refer to Figures 1 to 4. The present invention designs a collaborative energy management optimization method for fuel cell vehicles taking into account the air-conditioning system based on the soft-constrained actor-critic algorithm. Considering that the energy consumption changes of the air-conditioning system are usually ignored in the energy management of fuel cell vehicles, the main influencing factors of the temperature comfort in the vehicle cabin are analyzed, and the air-conditioning system model and the cabin heat load model are established. Taking hydrogen consumption, SOC maintenance and cabin temperature as optimization goals, a collaborative energy management optimization control framework taking into account the air-conditioning system is established by applying the soft-constrained actor-critic algorithm suitable for control tasks in a continuous action space, which realizes the collaborative control of energy management and air-conditioning system, and optimizes the hydrogen combustion economy and cabin temperature comfort of fuel cell vehicles. As shown in Figure 1, the energy management collaborative optimization method specifically includes the following steps:

S1: Obtain key parameter information of fuel cell vehicles, including:

The vehicle status parameter information includes: vehicle speed, cabin heat load parameters, motor operating efficiency and transmission system characteristic parameters;

Fuel cell parameter information includes: fuel cell power, efficiency, and hydrogen energy consumption;

The power battery parameter information includes: the power battery's state of charge, internal resistance, and open circuit voltage;

The air conditioning system parameter information includes: the air conditioning system cooling capacity/heating capacity and the corresponding power.

S2: Establish a fuel cell vehicle collaborative energy management model, as shown in Figures 2 and 3. The specific steps are:

S21: Establish the longitudinal dynamics model of the whole vehicle:

P _drive =(F _air +F _f +F _i +m ₀ a)·v

_Pdem ＝ _Pb + _Pfc · _ηDC/DC

Among them, _m0 represents the vehicle mass; v represents the vehicle speed; a represents the vehicle acceleration; F _air represents the air resistance; F _f represents the rolling resistance; _Fi represents the acceleration resistance; η _m , η _DC/AC , η _DC/DC and η _motor represent the transmission efficiency, DC/AC converter efficiency, DC/DC converter efficiency and motor efficiency respectively; P _drive , P _dem , P _b and P _fc represent the driving power at the vehicle wheels, the required power, the battery output power and the fuel cell output power respectively.

S22: Establish fuel cell model:

η _fc ＝f _η (P _fc )

分别表示为效率和氢能消耗量的拟合函数，可通过插值法计算效率与氢耗。Among them, f _η (·) and

They are respectively expressed as fitting functions of efficiency and hydrogen energy consumption, and the efficiency and hydrogen consumption can be calculated by interpolation method.

S23: Establishing power battery model:

Among them, I _L represents the power battery current; V _oc represents the power battery open circuit voltage; R _in represents the power battery equivalent internal resistance; SOC ₀ represents the initial SOC; Q _t represents the maximum capacity of the power battery; t ₀ represents the initial time; t _f represents the final time.

S24: Build the motor model:

η _m = f _m (ω _m , T _m )

Wherein, ω _m and T _m represent the motor speed and torque respectively; P _m represents the motor output power, and f _m (·) represents the fitting function of the motor working efficiency. The motor working efficiency can be obtained by interpolation method.

S25: Establish air conditioning system model:

Wherein, Q _ac represents the cooling capacity or heating capacity of the air-conditioning system; P _ac represents the corresponding power consumption of the air-conditioning system; η _cop represents the performance coefficient of the air-conditioning system.

S26: Establish cabin heat load model:

Q _c = ∑ KF (T _out -T _in )

Q _h =145+116n

Q _n ＝m _e ξCp _air (T _out -T _in )

Among them, _Qc , _Qr , _Qh and _Qn represent the heat conduction load, radiation heat load, heat generated by the occupants (according to experience, the heat generated by the driver is about 145W, and each passenger generates about 116W of heat) and ventilation system heat load respectively; K represents the heat transfer coefficient; F represents the heat transfer area of the corresponding shell; _Tout represents the ambient temperature; _Tin represents the cabin air temperature; η represents the permeability; I represents the solar light intensity; _Ai represents the area of the windshield, left and right side windows and rear window; _θi represents the solar incident angle; β represents the shadow factor; n represents the number of passengers in the car; _me represents the air mass passing through the evaporator; ξ represents the air recirculation coefficient; Cp _air represents the indoor air heat capacity; ρ _air and V _air represent the cabin air density and cabin volume respectively.

S3: Based on the SAC algorithm, a fuel cell vehicle collaborative energy management optimization control framework taking into account the air conditioning system is established to solve the multi-objective optimization problem including hydrogen fuel economy and cabin temperature comfort. As shown in Figure 3, the soft-constrained actor-critic algorithm is used to achieve collaborative control of energy management and air conditioning system, optimizing the hydrogen fuel economy and cabin temperature comfort of fuel cell vehicles, specifically:

S301: In order to reflect key environmental information, the power battery SOC, fuel cell output power P _fc , vehicle speed v, and air conditioning cooling/heating capacity Q _ac are set as state variables to construct a state space, which can be expressed as:

S＝{SOC, _Pfc ,v, _Qac }

和空调系统制冷/制热容量变化量

设置为动作变量，构建动作空间，可表示为：S302: The collaborative energy management of the air conditioning system should not only allocate the power of the power source, but also maintain the thermal comfort of the cabin temperature according to the change of the cooling/heating capacity of the air conditioning system. To this end, the change of the fuel cell output power

and the change in cooling/heating capacity of the air conditioning system

Set as action variable and construct action space, which can be expressed as:

S303: To ensure the cabin temperature comfort, the cabin temperature is maintained at about 24°C. For this purpose, the reward function should also include the optimization item of cabin temperature change. Therefore, the reward function is set as the weighted sum of the three indicators of hydrogen energy consumption, SOC change and cabin temperature change, which can be expressed as:

R＝-(ζ·fuel(t)+ψ·(SOC(t)-0.7) ² +γ·(T _in -24) ² )

Among them, ζ, Ψ, and γ are weight factors of each optimization item. The trade-off between hydrogen energy consumption and cabin temperature comfort is solved by adjusting the weight factors, thereby solving the multi-objective optimization problem; fuel(t) represents the hydrogen energy consumption at the current moment; SOC(t) represents the state of charge of the power battery at the current moment.

S304: Combined with the SAC algorithm to solve the multi-objective optimization problem in energy management, the action entropy value is introduced into the SAC algorithm to make the action output more dispersed, thereby improving the algorithm's exploration ability, ability to learn new tasks and stability. The entropy value is expressed as:

H(π(·|s _t ))=-logπ(·|s _t )

Among them, H is the entropy of strategy π(·|s _t ).

S305: During the solution process, the actor network in the agent takes the state s _t as input, outputs the mean and variance of the action Gaussian distribution, and generates the action a _t using the reparameterization technique:

函数输出均值和方差；

和

分别表示高斯分布的均值和方差。Where τ _t represents the noise signal sampled from the standard normal distribution;

The function outputs the mean and variance;

and

represent the mean and variance of the Gaussian distribution respectively.

中。S306: After executing action a _t , the vehicle environment feeds back a reward r _t to the agent and transfers to the next state s _t+1 , thereby generating the interaction data {s _t , a _t , r _t , s _t+1 } between the environment and the agent and storing it in the experience pool

middle.

S307: Randomly extract a small batch of experience samples from the experience pool. In order to avoid overestimation when maximizing the action state function value and further overestimation when using the own network to calculate the target, introduce an evaluation critic network with parameters θ ₁ , θ ₂ and a target critic network with parameters θ′ ₁ , θ′ _2. Select the target critic network to output a smaller action state function value as the target value. For a specific state s _t and action a _t , the update formula of the soft constraint action value function Q _soft (s _t, a _t ) in the SAC algorithm is as follows:

Among them, r represents the reward obtained by the vehicle; γ represents the discount factor; α represents the temperature coefficient.

与

之间的均方误差，表示为：S308: When updating the policy network, the evaluation critic network is updated by minimizing the loss function L(θ _i ), which is defined as

and

The mean square error between them is expressed as:

表示为评估评论家网络参数为θ_i时的评价函数，而

表为目标评论家网络参数为θ′_i时的评价函数。in,

It is expressed as the evaluation function for evaluating the critic network parameter θ _i , and

The table shows the evaluation function when the target critic network parameter is θ′ _i .

定义为：S309: The actor network parameter update is achieved by minimizing the KL divergence. The smaller the KL value, the smaller the difference between the rewards corresponding to the output actions, and the better the convergence effect of the strategy. The objective function of the actor network

Defined as:

表示当前时刻下车辆状态s_t、执行动作a_t时的数学期望函数，

表示当前状态为s_t时的策略函数，

表示为策略函数的参数。Where D _KL is the KL divergence calculation expression; Z(s _t ) is the partition function used to normalize the distribution;

represents the mathematical expectation function of the vehicle state s _t at the current moment and the action a _t ,

represents the policy function when the current state is s _t ,

Represented as parameters of the policy function.

S310: Update the actor network parameters according to the gradient descent method, expressed as:

表示为关于策略函数参数

的下降梯度，

表示为关于当前时刻t下执行动作a_t的下降梯度；in,

Expressed as the policy function parameters

The descent gradient of

It is expressed as the descent gradient of executing action a _t at the current time t;

S311: In the SAC algorithm system, the adjustment of the temperature coefficient α is crucial to the training effect of the SAC algorithm. In different reinforcement learning tasks and training periods, the value of the optimal temperature coefficient is different. In order to achieve automatic adjustment of the temperature coefficient, the optimal temperature coefficient of each step can be updated by solving the minimum value of the objective function in the optimization problem. The objective function is expressed as:

表示为依据策略函数π_t执行动作a_t时的数学期望函数，π_t(a_t|s_t)表示为策略函数，s_t表示为当前时刻t下燃料电池汽车所处的状态，a_t则表示为当前时刻t时依据策略函数执行的动作。Where H ₀ represents the predefined threshold of the minimum policy entropy.

is represented as the mathematical expectation function when action a _t is executed according to the strategy function π _t , π _t (a _t |s _t ) is represented as the strategy function, s _t is represented as the state of the fuel cell vehicle at the current time t, and a _t is represented as the action executed according to the strategy function at the current time t.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims (8)

1. A fuel cell vehicle learning type cooperative energy management method considering an air conditioning system, which is characterized by comprising the following steps:

s1: acquiring vehicle state parameter information, fuel cell parameter information, power cell parameter information and air conditioning system parameter information of a fuel cell automobile;

s2: establishing a fuel cell vehicle collaborative energy management model, comprising: a whole vehicle longitudinal dynamics model, a fuel cell model, a power cell model, a motor model, an air conditioning system model and a cabin thermal load model;

s3: establishing a fuel cell automobile collaborative energy management optimization control strategy considering an air conditioning system, solving a multi-objective optimization problem comprising hydrogen economy and cabin temperature comfort by combining an SAC algorithm, and controlling the change of refrigerating/heating capacity of an air conditioner to maintain the cabin temperature in a comfort zone while performing energy flow optimization control; the SAC algorithm is a soft constraint actor commentator algorithm; the method for establishing the fuel cell automobile collaborative energy management optimization control strategy considering the air conditioning system specifically comprises the following steps:

s301: determining a state space: SOC of power battery and output power P of fuel battery _fc Vehicle speed v, refrigerating/heating capacity Q of air conditioning system _ac Set as state variables, construct a state space S, denoted as:

S＝{SOC,P _fc ,v,Q _ac }

s302: determining an action space: variation of fuel cell output power ∈P _fc And the variation of the refrigerating/heating capacity of the air conditioning system (Q) _ac Set as action variables, construct action space a, denoted as:

A＝{▽P _fc ,▽Q _ac }

s303: establishing a reward function: the bonus function R is set as a weighted sum of three indicators of hydrogen energy consumption, SOC variation and cabin temperature variation, expressed as:

R＝-(ζ·fuel(t)+ψ·(SOC(t)-0.7) ² +γ·(T _in -24) ² )

zeta, ψ and gamma are weight factors of each optimization term, and the balance problem between hydrogen energy consumption and cabin temperature comfort is solved by adjusting the weight factors, so that the multi-objective optimization problem is solved; fuel (t) represents the hydrogen energy consumption at the current time; SOC (t) represents the state of charge of the power battery at the current time; t (T) _in Expressed as cabin air temperature;

solving a multi-objective optimization problem comprising hydrogen economy and cabin temperature comfort by combining with a SAC algorithm, specifically comprising the following steps:

s311: solving a multi-objective optimization problem in energy management by combining a SAC algorithm, introducing motion entropy values into the SAC algorithm to enable motion output to be more dispersed, and further improving exploration capacity, new task learning capacity and stability of the algorithm, wherein the entropy values are expressed as:

H(π(·|s _t ))＝-logπ(·|s _t )

wherein H is the strategy pi (|s) _t ) Entropy of (2);

s312: during the solving process, the actor network in the agent is in the state s _t As input, inputYielding the mean and variance of the gaussian distribution of the motion, generating motion a using a re-parameterization technique _t ：

wherein ,τ_t Representing noise signals sampled from a standard normal distribution;

representing the mean and variance of the function output;

And

mean and variance of gaussian distribution are shown, respectively;

s313: executing action a _t Thereafter, the vehicle environment feeds back the reward r to the agent _t And transitions to the next state s _t+1 I.e. generating interaction data { s } of the environment and the agent _t ,a _t ,r _t ,s _t+1 And store in experience pool

In (a) and (b);

s314: randomly extracting small-batch experience samples from an experience pool, and introducing parameters theta ₁ ,θ ₂ Is a critics network and parameter θ ₁ ′，θ ₂ The target critics network selects a smaller action state function value output by the target critics network as a target value; for a specific state s _t And action a _t Soft constraint action value function Q in SAC algorithm _soft (s _t ,a _t ) The update formula is as follows:

wherein r represents a reward earned by the vehicle; gamma represents a discount factor; alpha represents a temperature coefficient;

s315: when updating the policy network, the policy network is updated by minimizing the loss function L (θ _i ) Updating an evaluation critic network, the loss function being defined as

And->

The mean square error between them is expressed as:

wherein ,

representing evaluating critics network parameters as θ _i Evaluation function at time->

Representing the network parameter of the target critics as theta _i ' evaluation function at time;

s316: actor network parameter updating is achieved by minimizing KL divergence; objective function of actor network

The definition is as follows:

wherein ,D_KL Representing a KL divergence calculation expression; z(s) _t ) Is a distribution function for normalizing the distribution;

representing the state s of the vehicle at the current time _t Executing action a _t Mathematical expectation function of time;

Representing the current state as s _t Policy function at time->

Parameters expressed as policy functions;

s317: updating actor network parameters according to a gradient descent method, wherein the actor network parameters are expressed as follows:

wherein ,

expressed as about policy function parameters->

Gradient of decline of->

Represented as action a being performed in relation to the current time t _t Is a decreasing gradient of (2);

s318: the optimal temperature coefficient of each step can be obtained by updating the minimum value of the objective function in the optimization problem, and the objective function is expressed as:

wherein ,H₀ A threshold representing a predefined minimum policy entropy,

represented as a function of policy pi _t Executing action a _t Mathematical expectation function of time, pi _t (a _t |s _t ) Expressed as a policy function, s _t Is expressed as the state of the fuel cell automobile at the current time t, a _t Then this is denoted as the action performed according to the policy function at the current time t.

2. The fuel cell vehicle learning collaborative energy management method according to claim 1, wherein in step S1, the vehicle state parameter information includes: vehicle speed, cabin thermal load parameters, motor operating efficiency and transmission system characteristic parameters; the fuel cell parameter information includes: power, efficiency, and hydrogen energy consumption of the fuel cell; the power battery parameter information includes: the state of charge, internal resistance and open circuit voltage of the power battery; the air conditioning system parameter information includes: air conditioning system cooling capacity/heating capacity and corresponding power.

3. The fuel cell vehicle learning collaborative energy management method according to claim 1, wherein in step S2, a vehicle longitudinal dynamics model is established as follows:

P _drive ＝(F _air +F _f +F _i +m ₀ a)·v

P _dem ＝P _b +P _fc ·η _DC/DC

wherein ,m₀ Representing the quality of the whole vehicle; v is the speed of the whole vehicle; a represents vehicle acceleration; f (F) _air Expressed as air resistance; f (F) _f Expressed as rolling resistance; f (F) _i Expressed as acceleration resistance; η (eta) _m 、η _DC/AC 、η _DC/DC η _motor Respectively representing transmission efficiency, DC/AC converter efficiency, DC/DC converter efficiency and motor efficiency; p (P) _drive 、P _dem 、P _b P _fc Respectively representing the driving power at the wheels of the vehicle, the required power, the battery output power, and the fuel cell output power.

4. The fuel cell vehicle learning collaborative energy management method according to claim 3 wherein in step S2, a fuel cell model is established as:

η _fc ＝f _η (P _fc )

wherein ,f_η(·) and

Expressed as a fitted function of efficiency and hydrogen energy consumption, respectively, the efficiency and hydrogen consumption were calculated by interpolation.

5. The fuel cell vehicle learning collaborative energy management method according to claim 3, wherein in step S2, a power cell model is established as follows:

wherein ,I_L Expressed as power cell current; v (V) _oc Expressed as power cell open circuit voltage; r is R _in Expressed as the equivalent internal resistance of the power battery;SOC ₀ denoted as initial SOC; q (Q) _t Expressed as power cell maximum capacity; t is t ₀ Denoted as initial time; t is t _f Represented as the final time.

6. The fuel cell vehicle learning collaborative energy management method according to claim 3, wherein in step S2, a motor model is built as follows:

η _m ＝f _m (ω _m ,T _m )

wherein ,ω_m and T_m Respectively representing the motor rotation speed and the motor torque; p (P) _m Expressed as motor output power, f _m (. Cndot.) represents a fitting function of the motor working efficiency, which is obtained by interpolation.

7. The fuel cell vehicle learning collaborative energy management method according to claim 1, wherein in step S2, an air conditioning system model is established as follows:

wherein ,Q_ac Expressed as cooling capacity or heating capacity of the air conditioning system; p (P) _ac Expressed as the corresponding power consumption of the air conditioning system; η (eta) _cop Expressed as an air conditioning system coefficient of performance.

8. The fuel cell vehicle learning collaborative energy management method according to claim 1, wherein in step S2, a vehicle cabin thermal load model is established as follows:

Q _c ＝∑KF(T _out -T _in )

Q _h ＝145+116n

Q _n ＝m _e ξCp _air (T _out -T _in )

wherein ,Q_c 、Q _r 、Q _h and Q_n respectively representing heat conduction load, radiation heat load, heat generated by personnel in the vehicle and heat load of a ventilation system; k is expressed as a heat transfer coefficient; f represents the heat transfer area of the corresponding housing; t (T) _out Expressed as ambient temperature;

T _in expressed as cabin air temperature; η is expressed as permeability; i is expressed as the intensity of sunlight; a is that _i Represented as windshield, left and right side windows, and rear window area; θ _i Expressed as the incident angle of sunlight; beta is denoted as a shading factor; n represents the number of passengers in the vehicle;

m _e represented as the mass of air passing through the evaporator; ζ is the air recirculation coefficient; cp _air Expressed as indoor air heat capacity; ρ _air and V_air Respectively as the air density in the cabin and the cabin volume.

Images