More Web Proxy on the site http://driver.im/

research-article

Advanced optimal tracking integrating a neural critic technique for asymmetric constrained zero-sum games

Authors:

Junfei QiaoAuthors Info & Claims

Volume 177, Issue C

https://doi.org/10.1016/j.neunet.2024.106388

Published: 01 September 2024 Publication History

Abstract

This paper investigates the optimal tracking issue for continuous-time (CT) nonlinear asymmetric constrained zero-sum games (ZSGs) by exploiting the neural critic technique. Initially, an improved algorithm is constructed to tackle the tracking control problem of nonlinear CT multiplayer ZSGs. Also, we give a novel nonquadratic function to settle the asymmetric constraints. One thing worth noting is that the method used in this paper to solve asymmetric constraints eliminates the strict restriction on the control matrix compared to the previous ones. Further, the optimal controls, the worst disturbances, and the tracking Hamilton–Jacobi–Isaacs equation are derived. Next, a single critic neural network is built to estimate the optimal cost function, thus obtaining the approximations of the optimal controls and the worst disturbances. The critic network weight is updated by the normalized steepest descent algorithm. Additionally, based on the Lyapunov method, the stability of the tracking error and the weight estimation error of the critic network is analyzed. In the end, two examples are offered to validate the theoretical results.

References

[1]

Arogeti S.A., Lewis F.L., Static output-feedback H ∞ control design procedures for continuous-time systems with different levels of model knowledge, IEEE Transactions on Cybernetics 53 (3) (2023) 1432–1446.

[2]

Huang Y., Liu D., Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithm, Neurocomputing 125 (2014) 46–56.

[3]

Huo X., Karimi H.R., Zhao X., Wang B., Zong G., Adaptive-critic design for decentralized event-triggered control of constrained nonlinear interconnected systems within an identifier-critic framework, IEEE Transactions on Cybernetics 52 (8) (2022) 7478–7491.

[4]

Huo Y., Wang D., Li M., Qiao J., Decentralized event-triggered asymmetric constrained control through adaptive critic designs for nonlinear interconnected systems, IEEE Transactions on Systems, Man, and Cybernetics: Systems 54 (1) (2024) 391–402.

[5]

Huo Y., Wang D., Qiao J., Li M., Adaptive critic design for nonlinear multi-player zero-sum games with unknown dynamics and control constraints, Nonlinear Dynamics 111 (2023) 11671–11683.

[6]

Jiang H., Zhang H., Han J., Zhang K., Iterative adaptive dynamic programming methods with neural network implementation for multi-player zero-sum games, Neurocomputing 307 (2018) 54–60.

[7]

Kim J., Yang I., Maximum entropy optimal control of continuous-time dynamical systems, IEEE Transactions on Automatic Control 68 (4) (2023) 2018–2033.

[8]

Lewis F.L., Jagannathan S., Yesildirek A., Neural network control of robot manipulators and nonlinear systems, Taylor & Francis, London, U.K., 1999.

[9]

Li M., Wang D., Qiao J., Neural critic learning for tracking control design of constrained nonlinear multi-person zero-sum games, Neurocomputing 512 (2022) 456–465.

[10]

Li M., Wang D., Zhao M., Qiao J., Event-triggered constrained neural critic control of nonlinear continuous-time multiplayer nonzero-sum games, Information Sciences 631 (2023) 412–428.

[11]

Liang M., Liu D., Liquid-updating impulsive adaptive dynamic programming for continuous nonlinear systems, IEEE Transactions on Systems, Man, and Cybernetics: Systems 54 (2) (2024) 716–728.

[12]

Liu D., Li H., Wang D., Online synchronous approximate optimal learning algorithm for multiplayer nonzero-sum games with unknown dynamics, IEEE Transactions on Systems, Man, and Cybernetics: Systems 44 (8) (2014) 1015–1027.

[13]

Liu D., Xue S., Zhao B., Luo B., Wei Q., Adaptive dynamic programming for control: A survey and recent advances, IEEE Transactions on Systems, Man, and Cybernetics: Systems 51 (1) (2021) 142–160.

[14]

Modares H., Lewis F.L., Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning, Automatica 50 (7) (2014) 1780–1792.

Digital Library

[15]

Modares H., Lewis F.L., Jiang Z.-P., H ∞ Tracking control of completely unknown continuous-time systems via off-policy reinforcement learning, IEEE Transactions on Neural Networks and Learning Systems 26 (10) (2015) 2550–2562.

[16]

Qiao J., Li M., Wang D., Asymmetric constrained optimal tracking control with critic learning of nonlinear multiplayer zero-sum games, IEEE Transactions on Neural Networks and Learning Systems 35 (4) (2024) 5671–5683.

[17]

Qiao J., Zhao M., Wang D., Ha M., Adjustable iterative Q-learning schemes for model-free optimal tracking control, IEEE Transactions on Systems, Man, and Cybernetics: Systems 54 (2) (2024) 1202–1213.

[18]

Schwerdtner P., Voigt M., Fixed-order H-infinity controller design for port-Hamiltonian, Automatica 152 (2023).

[19]

Song R., Wei Q., Zhang H., Lewis F.L., Discrete-time non-zero-sum games with completely unknown dynamics, IEEE Transactions on Cybernetics 51 (6) (2021) 2929–2943.

[20]

Tang Y., Yang X., Robust tracking control with reinforcement learning for nonlinear-constrained systems, International Journal of Robust and Nonlinear Control 32 (18) (2022) 9902–9919.

[21]

Vamvoudakis K.G., Lewis F.L., Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton–Jacobi equations, Automatica 47 (8) (2011) 1556–1569.

[22]

Wang D., Event-based iterative neural control for a type of discrete dynamic plant, Chinese Journal of Engineering 44 (3) (2022) 411–419.

[23]

Wang D., Gao N., Liu D., Li J., Lewis F.L., Recent progress in reinforcement learning and adaptive dynamic programming for advanced control applications, IEEE/CAA Journal of Automatica Sinica 11 (1) (2024) 18–36.

[24]

Wang D., Hu L., Zhao M., Qiao J., Dual event-triggered constrained control through adaptive critic for discrete-time zero-sum games, IEEE Transactions on Systems, Man, and Cybernetics: Systems 53 (3) (2023) 1584–1595.

[25]

Wang D., Li X., Zhao M., Qiao J., Adaptive critic control design with knowledge transfer for wastewater treatment applications, IEEE Transactions on Industrial Informatics 20 (2) (2024) 1488–1497.

[26]

Wang D., Liu D., Zhang Y., Li H., Neural network robust tracking control with adaptive critic framework for uncertain nonlinear systems, Neural Networks 97 (2018) 11–18.

[27]

Wang D., Ren J., Ha M., Qiao J., System stability of learning-based linear optimal control with general discounted value iteration, IEEE Transactions on Neural Networks and Learning Systems 34 (9) (2023) 6504–6514.

[28]

Wang D., Zhao H., Li X., Adaptive critic control for wastewater treatment systems based on multiobjective particle swarm optimization, Chinese Journal of Engineering 46 (5) (2024) 908–917.

[29]

Wei Q., Zhou T., Lu J., Liu Y., Su S., Xiao J., Continuous-time stochastic policy iteration of adaptive dynamic programming, IEEE Transactions on Systems, Man, and Cybernetics: Systems 53 (10) (2023) 6375–6387.

[30]

Werbos P.J., Beyond regression: New tools for prediction and analysis in the behavioral sciences, [Ph.D. dissertation] Harvard University, Cambridge, MA, USA, 1974.

[31]

Xue S., Luo B., Liu D., Event-triggered adaptive dynamic programming for unmatched uncertain nonlinear continuous-time systems, IEEE Transactions on Neural Networks and Learning Systems 32 (7) (2021) 2939–2951.

[32]

Yang X., Gao Z., Zhang J., Event-driven H ∞ control with critic learning for nonlinear systems, Neural Networks 132 (2020) 30–42.

[33]

Yang X., He H., Self-learning robust optimal control for continuous-time nonlinear systems with mismatched disturbances, Neural Networks 99 (2018) 19–30.

[34]

Yang X., He H., Event-driven H ∞-constrained control using adaptive critic learning, IEEE Transactions on Cybernetics 51 (10) (2021) 4860–4872.

[35]

Yang X., He H., Zhong X., Approximate dynamic programming for nonlinear-constrained optimizations, IEEE Transactions on Cybernetics 51 (5) (2021) 2419–2432.

[36]

Yang X., Xu M., Wei Q., Approximate dynamic programming for event-driven H ∞ constrained control, IEEE Transactions on Systems, Man, and Cybernetics: Systems 53 (9) (2023) 5922–5932.

[37]

Yang X., Zhou Y., Dong N., Wei Q., Adaptive critics for decentralized stabilization of constrained-input nonlinear interconnected systems, IEEE Transactions on Systems, Man, and Cybernetics: Systems 52 (7) (2022) 4187–4199.

[38]

Yang X., Zhou Y., Gao Z., Reinforcement learning for robust stabilization of nonlinear systems with asymmetric saturating actuators, Neural Networks 158 (2023) 132–141.

[39]

Yu S., Zhang H., Ming Z., Sun J., Optimal control for continuous-time unknown nonlinear affine systems: A Q-learning approach, IEEE Transactions on Automation Science and Engineering (2023),. Early access.

[40]

Zhang H., Ming Z., Yan Y., Wang W., Data-driven finite-horizon H ∞ tracking control with event-triggered mechanism for the continuous-time nonlinear systems, IEEE Transactions on Neural Networks and Learning Systems 34 (8) (2023) 4687–4701.

[41]

Zhang K., Zhang H., Cai Y., Su R., Parallel optimal tracking control schemes for mode-dependent control of coupled Markov jump systems via integral RL method, IEEE Transactions on Automation Science and Engineering 17 (3) (2020) 1332–1342.

[42]

Zhang K., Zhang H., Jiang H., Wang Y., Near-optimal output tracking controller design for nonlinear systems using an event-driven ADP approach, Neurocomputing 309 (2018) 168–178.

Digital Library

[43]

Zhang S., Zhao B., Liu D., Zhang Y., Observer-based event-triggered control for zero-sum games of input constrained multi-player nonlinear systems, Neural Networks 144 (2021) 101–112.

[44]

Zhao M., Wang D., Qiao J., Ha M., Ren J., Advanced value iteration for discrete-time intelligent critic control: A survey, Artificial Intelligence Review 56 (10) (2023) 12315–12346.

[45]

Zhao H., Zong G., Zhao X., Wang H., Xu N., Zhao N., Hierarchical sliding-mode surface-based adaptive critic tracking control for nonlinear multiplayer zero-sum games via generalized fuzzy hyperbolic models, IEEE Transactions on Fuzzy Systems 31 (11) (2023) 4010–4023.

Recommendations

Neural critic learning for tracking control design of constrained nonlinear multi-person zero-sum games
Abstract
In this paper, an adaptive critic method based on neural networks is established to solve the tracking control problem for multi-person zero-sum games with constrained nonlinear dynamics. First, an augmented system is constructed with the ...
Event-triggered constrained neural critic control of nonlinear continuous-time multiplayer nonzero-sum games
Abstract
In this article, by exploiting the event-triggered neural critic control scheme, the optimal control question of nonlinear continuous-time multiplayer nonzero-sum games with asymmetric control constraints is investigated. First, to satisfy the ...
Tracking control optimization scheme of continuous-time nonlinear system via online single network adaptive critic design method

In this paper, the optimal tracking control problem (OTCP) for a class of continuous-time nonlinear systems with infinite horizon cost is discussed. An online adaptive critic design method is proposed to learn the solution of OTCP by constructing an ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neural Networks

Neural Networks Volume 177, Issue C

Sep 2024

298 pages

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Science Ltd.

United Kingdom

Publication History

Published: 01 September 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents