DE102020106880A1 - Method and device for calibrating a control and regulating device system for controlling and regulating an operating state - Google Patents

Abstract

The invention provides a method for calibrating a control and regulating device system for controlling and regulating an operating state with the following features: reading in input variables; Calculation of at least one control parameter in accordance with the control and regulating device model using the input variables and model parameters (100); Modifying at least one of the model parameters (101); Measuring at least one control parameter (102); Comparison of the calculated control parameter with the correspondingly measured control parameter (103); Using at least one policy to select an optimal action in a state; Evaluation of the action as positive, negative or neutral depending on the comparison (103) and improvement of the guideline on the basis of the evaluation by means of reinforcement learning. The invention also provides a corresponding device, a corresponding computer program and a corresponding storage medium.

Description

The present invention relates to a method for calibrating a control and regulating device system for controlling and regulating an operating state. The present invention also relates to a corresponding device, a corresponding computer program and a corresponding storage medium.

It is known that injectors deliver individual amounts of fuel to each cylinder in an engine. An injection pump is a metering pump that is designed for high pressure and forms part of the injection system in internal combustion engines. The injection pump provides a defined amount of fuel per stroke with the necessary pressure (injection pressure) to deliver the fuel through the injection valve into the combustion chamber or the intake pipe.

One engine has one injection valve per cylinder. The injection nozzles atomize a very fine fuel mist either in the intake tract (intake manifold injection) or directly in the combustion chamber of the cylinder (direct injection). Modern engines also offer a combination of direct and manifold injection. Depending on the load range, intake manifold and direct injection alternate with the aim of reducing the formation of nitrogen oxides. This precisely metered atomization of the fuel by the injection nozzles results in an optimized combustion of the fuel, which results in better engine performance. In order to achieve such fine atomization, the fuel must be injected with overpressure. In the case of manifold injection, where the injection nozzles are supplied with fuel via a common injection rail, the injection pressure is around three to six bar. With direct injection, where the injection nozzles are fed via a high-pressure pump, it is up to 350 bar Parameters for the combustion quality of the engine.

There are primarily two approaches known for determining and predicting the injection pressure in an internal combustion engine. The first approach is based on a physical model, in which a target injection pressure is calculated as a function of various parameters such as the amount of fuel, the temperature of the engine, the current load, etc. The corresponding values depending on the state of the engine are stored in the form of tables, characteristic curves and diagrams, preferably in an electronic control unit of the vehicle.

Another approach uses regression analysis to model and predict the target injection pressure. Regression methods are statistical analysis methods that aim to model relationships between a dependent and one or more independent variables. The regression models receive, for example, the amount of fuel, the fuel charge, the number of revolutions or the temperature of the engine as input and calculate an estimated value for the injection pressure using mathematical methods such as the method of least squares or neural networks.

However, these tables and diagrams or the mathematical procedures used are based on expert knowledge and are no longer changed after they have been implemented. Furthermore, they are not adapted for an individual engine, but are defined for a specific model series.

The DE 10 2010 052 857 A1 describes a system and method for real-time self-learning characterization of fuel injector performance during engine operation. The system includes an algorithm for an engine controller that enables the controller to learn the correlation between fuel mass and pulse width for each injector in the engine in real time while the engine is running. The controller gradually detects the pulse widths that reach the desired fuel mass, while continuously reading values such as B. can adjust the temperature and fuel rail pressure to what he has learned based on various input changes. The controller then uses the learned actual power of each injector to command the pulse width required to achieve the desired amount of fuel for each cylinder in each cycle.

The WO 2009/019663 A2 describes a fuel injector for an internal combustion engine, the fuel injector comprising a fuel supply passage containing fuel under high pressure and a pressure sensor for measuring the fuel pressure. Using a hydraulic behavior profile to predict fuel pressure, a control signal is generated to control the amount of fuel injected during the injection event in accordance with the predicted fuel pressure.

The US 6,701,905 B1 describes a fuel pressure control for a fuel engine having an algorithm that includes adaptively learned corrections for fuel injection pulse width to dynamically adjust a base fuel pressure control signal. The controller dynamically adjusts the fuel pressure in such a way that a The air / fuel mixing ratio is optimized instead of controlling a predetermined fuel pressure.

DE 10 2012 201 767 A1 relates to a method for monitoring the dynamics of gas sensors of an internal combustion engine, the gas sensors having a low-pass behavior depending on geometry, measuring principle, aging or contamination, with a dynamic diagnosis when the gas state variable to be measured changes based on a comparison of a modeled and a measured signal is carried out and wherein the measured signal is an actual value of an output signal of the gas sensor and the modeled signal is a model value. It is provided that the parameters of the low-pass behavior are determined depending on the direction by minimizing direction-dependent error signals, which are formed by high-pass filtering and linking with direction-dependent saturation characteristics, the direction-dependent error signals by comparing the modeled and the measured signal for an increasing and a decreasing signal component be calculated. The method is intended to determine parameters of gas sensors, such as a time constant, a dead time or an amplification factor or any combination thereof, depending on the direction, ie for both rising and falling signals. This should be used particularly advantageously in exhaust gas probes designed as continuous lambda probes, without the need to intervene in the air or fuel system of the internal combustion engine.

The object on which the invention is based is to create a method and a system for automatically calibrating a control and regulating device for regulating the injection pressure in an internal combustion engine, which is characterized by high reliability and accuracy and can be easily implemented.

The invention provides a method for calibrating a control and regulating device system for controlling and regulating an operating state, a corresponding device, a corresponding computer program and a corresponding storage medium according to the independent claims.

One advantage of this solution is its suitability for models and controls in a wide variety of application areas. Combustion engines, electrical machines and batteries as well as functions for their diagnosis, aircraft, fuel cells, hybrid drives, trains, boats, robots, motorcycles, electric cars and other road vehicles come into consideration. Think, for example, of driving dynamics controller (aerodynamics, suspension, chassis, steering, etc.), safety systems (alarm, central locking, parking brake, etc.), driver assistance (blind spot monitoring, adaptive cruise control, lane departure warning, crosswind stabilization, etc.), equipment in the passenger compartment (seats, air conditioning , Panoramic roof, cameras, etc.) and other additional systems (brake booster, front axle jack, slope, tire pressure control, windshield wipers, headlights, etc.).

Further advantageous refinements of the invention are specified in the dependent claims.

Figure list

An embodiment of the invention is shown in the drawings and is described in more detail below.

• 1 eine erste schematische Blockdarstellung eines Verfahrens nach einer Ausführungsform der Erfindung;
• 2 eine zweite schematische Blockdarstellung des Verfahrens;
• 3 mehrere mit einem zentralen Datenspeicher verbundene Steuer- und Regelvorrichtungssysteme; und
• 4 die Bewertung nachfolgender Modifizierungen anhand eines Unterschiedes zwischen einem berechneten und gemessenen oder simulierten Steuerungs-Parameter.

It shows

• 1 a first schematic block diagram of a method according to an embodiment of the invention;
• 2 a second schematic block diagram of the method;
• 3 a plurality of control and regulating device systems connected to a central data store; and
• 4th the evaluation of subsequent modifications based on a difference between a calculated and a measured or simulated control parameter.

1 illustrates the basic sequence of a method according to the invention. In step 100 control parameters are calculated according to the control and regulating device model. This is done using input variables and model parameters of the control and regulating device system. Operating states of an internal combustion engine, for example, can be controlled with the control parameters. The control parameters include, for example, the position of a throttle valve, the air-fuel ratio in a combustion chamber or the state of an injection pump, for example a piston movement or the speed of rotation of a pump shaft of the injection pump.

The input variables include values and / or data which are preferably simulated or which are provided by sensors, characteristic maps or other means for recording operating states of the internal combustion engine or environmental information, an injected fuel quantity, an engine speed, a composition of a fuel mixture, a temperature of a cylinder or a Lambda value of a lambda probe. Conceivable inputs and outputs each include Depending on the type of model, longitudinal, lateral and rotational speed, acceleration, vibration, brake and accelerator pedal position, angle, phase, deviation between target and actual value, mass flow, efficiency, energy, enthalpy of media (liquids, gases or vapors), Temperature or pressure of media or components, humidity, density, position and in particular height, duty cycle, current strength, voltage, capacity, bit switch (true / false), schedule, torque, power, frequency, state of charge or health as well as service life of a battery, luminosity , Power, number of battery stacks or cells, incline, inserted, selected or specified gear stage of a multi-step transmission or other key figures. Derived input variables, for example from images from cameras, optical or acoustic waves, GPS signals, inverter gates or electrical loads can also be considered.

The step 100 can be called an agent if the in 1 The presented method is understood as reinforcing machine learning.

The learning reinforcement agent 100 contains calculation methods and algorithms f _i for mathematical regression methods or physical model calculations, which returns an action for a given state based on a guideline and improves this guideline based on rewards. An action can, for example, be a modification of at least one of the model parameters. A state is a representation of the calibration status and can, for example, contain information about the deviation between calculated and measured / simulated control parameters. The mathematical functions f _t can be mean values, minimum and maximum values, Fast Fourier Transformations, integral and differential calculations, linear regression methods, Markov methods, probability methods such as Monte Carlo methods, temporal difference learning, as Q-learning or act as DYNAQ or as POMDP or as SARSA but also extended Kalman filters, radial basic functions, data fields, convergent neural networks, artificial neural networks and / or feedback neural networks.

In step 101 several or one of the model parameters are modified. Each parameter can be increased, decreased or left unchanged.

The step 101 can be understood as the agent's action in reinforcing machine learning.

In step 102 a control parameter is read in. This can be done using a sensor, for example, or a simulated signal.

In step 103 the calculated control parameter is compared with the measured or simulated control parameter. Depending on the comparison and on potential damage, the modification carried out is rated as positive, negative or neutral. After step 103 will step again 100 carried out, which is then followed by the steps 101 to 103 connect.

If the modification in step 103 was rated as positive, the probability of the previous action being selected for this state in the policy can be increased. If the modification in step 103 however, was rated as negative, the probability of the previous action being selected for this state in the policy will be reduced. When the policy has converged, it will always return the optimal action for a given state.

The method can be ended, for example, when the comparison in step 103 the difference between the calculated control parameter and the measured / simulated control parameter is in a range between 1-10%.

In 2 the procedure is over 1 shown again. It is also a central data store 200 presented in what information about the in step 101 modifications made and via the in step 103 The comparison carried out and the evaluations of the modifications are saved. The central data storage 200 can for example be arranged remotely from the control and regulating device system. Access to the central data store 200 can be done via the Internet, for example.

The storage of information in the central data store 200 is beneficial to keep in step 101 the agent carries out the best possible modification by retrieving information from the central data store before carrying out the modification 200 reads about past modifications. The information can then be used to carry out a modification that is very likely to be evaluated as positive.

In 3 is shown as several control and regulating device systems 300 to 302 who have all that in the 1 and 2 Carry out the procedures shown, information in the central data store 200 save and retrieve from it. In this way, the wealth of experience that each of the control and regulating device systems 300 to 302 can access, significantly enlarged. So with Modifications evaluated as positive are more likely to be carried out, so that the most efficient calibration possible is achieved. This procedure can also be referred to as reinforcement learning (RL) with several agents.

The design of the reward function used within the framework of the RL poses a particular challenge in practical application. The reward function is usually a linear combination of several criteria, so-called characteristics, the weights of which have to be weighed against each other. It is critical that the reward function captures the various criteria associated with the desired behaviors and required for an agent to perform an appropriate calibration. For this reason, practical applications of the RL are not straightforward and require the solution of the optimization problem associated with the design of such a reward function.

The reward function is conveniently based on the observed behavior of an expert, i. H. of an engineer, scientist or the like. The preferred approach here is not to capture the expert's strategy as such, but rather to find out which reward function the expert is trying to maximize. In other words, it aims to understand the expert's intentions and their respective roles in the calibration.

The person skilled in the art is familiar with the resulting optimization problem of adapting the weights of the individual features of the reward function so that the reward function is maximized based on the expert's approach and can be distinguished from suboptimal specifications as inverse reinforcement learning (IRL).

For this purpose, it is recorded, for example, which action - in the present application example, increasing, maintaining or reducing a model parameter - a real expert performs at each stage of the calibration process, as well as the inputs and the measured features of the system, for example the deviation between the modeled and real output, damage the hardware or temperature. Since the expert can only cope with a limited number of states, it can prove to be expedient to generalize his behavior by extrapolating the recorded samples, which, as it were, embody the “trajectory” of the expert.

An optimization method according to the invention modifies the weights of the reward function so that the reward of this trajectory of the expert is maximized. The optimization function also selects those weights that maximize the difference between the expert's strategy and alternative, sub-optimal strategies. For example, the maximum entropy method (MEM), Bayesian update, maximum margin or other classifiers as well as Gaussian process or other regression can be used for this purpose.

In 4th is shown how - based on a first difference 400 Subsequent modifications between the calculated control parameter and the measured / simulated control parameter are evaluated using a reward function defined in this way. First is in step 101 modified one of the model parameters in three different ways. A + stands for an increase in the model parameter, A = 0 for a retention and A- for a decrease in the model parameter. After the modification A +, there will be less difference 401 was found between the calculated control parameter and the measured / simulated control parameter, whereas the modification A = 0 shows the same difference 400 and with modification A- a greater difference 403 is detected.

On this basis, the modification A + becomes from step 101 as positive, the modification A = 0 as neutral and the modification A- as negative. The model parameters modified after the modification A + are then used as the starting point for a new modification 101 ' used. There is a smaller difference with the modification A = 0 402 than the difference 401 was found between the calculated control parameter and the measured / simulated control parameter, whereas with the modification A + the difference 401 remains the same and with modification A-the difference 404 bigger than the difference 401 .

Based on this, the modification A = 0 from step 101 ' as positive, the modification A + as neutral and the modification A- as negative. The model parameters modified after the modification A- are then used as the starting point for a new modification 101 " used. There is a difference with the modification A- 405 determined between the calculated control parameter and the measured / simulated control parameter, which is less than a threshold value of, for example, 5% and in particular 1% and less than the difference 402 is. Those found after the modifications A = 0 and A + differences 406 are, however, greater than the difference 402 so that they are discarded. Because the difference 405 is less than the threshold value, the calibration process is ended at this point.

It should be noted that instead of the reinforcement learning described, imitation learning or other machine learning (ML) approaches may also be used without departing from the scope of the invention.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102010052857 A1 [0007]
• WO 2009/019663 A2 [0008]
• US 6701905 B1 [0009]
• DE 102012201767 A1 [0010]

Claims (10)

Method for calibrating a control and regulating device system (300) for controlling and regulating an operating state, the control and regulating device system being designed to use a control and regulating device model (300) with model parameters for controlling and regulating the operating state , characterized by the following steps: - reading in measured or simulated input variables; - Calculation of at least one control parameter according to the control and regulating device model using the input variables and model parameters (100); - Modification of at least one of the model parameters (101); - Reading in at least one measured or simulated control parameter (102); - Comparison of the calculated control parameter with the correspondingly read in control parameter (103); Use of at least one policy to select an optimal action in a state; - Evaluation of the action as positive, negative or neutral depending on the comparison (103) and - Improvement of the guideline on the basis of the evaluation by means of reinforcement learning. Procedure according to Claim 1 , characterized by the following features: - in the comparison (103) a difference (400-406) between the calculated control parameter and the correspondingly measured control parameter is determined and - the method is terminated when the difference (400-406) a falls below the specified threshold. Procedure according to Claim 2 , characterized by the following feature: - the threshold is specified as a proportional factor of the calculated or measured control parameter. Procedure according to Claim 3 , characterized by the following feature: - the proportion factor is between 1 and 10%. Method according to one of the Claims 1 to 4th , characterized by the following features: - information about the modification (101), the comparison (103) and the evaluation are stored in a data memory (200) and - before the modification (101) the information is read from the data memory (200). Procedure according to Claim 5 , characterized by the following feature: - the control and regulating device system (300) accesses the data memory (200) via a computer network. Procedure according to Claim 6 , characterized by the following feature: the information is shared with further control and regulating device systems (301, 302) via the computer network. Device for calibrating a control and regulating device system for controlling and regulating an operating state, the control and regulating device system being designed to use a control and regulating device model with model parameters for controlling and regulating the operating state, characterized by the following features: - means for measuring input quantities; - Means for calculating at least one control parameter in accordance with the control and regulating device model using the input variables and model parameters; - Means for modifying at least one of the model parameters; - Means for measuring at least one control parameter; - Means for comparing the calculated control parameter with the correspondingly measured control parameter; - Means for evaluating the modification as positive or negative depending on the comparison. Computer program which is set up to carry out all steps of a method according to one of the Claims 1 to 7th perform. Machine-readable storage medium with a computer program stored thereon Claim 9 .

Images