More Web Proxy on the site http://driver.im/

research-article

Reinforcement learning for robust stabilization of nonlinear systems with asymmetric saturating actuators

Authors:

Yingjiang Zhou,

Zhongke GaoAuthors Info & Claims

Volume 158, Issue C

Pages 132 - 141

https://doi.org/10.1016/j.neunet.2022.11.012

Published: 01 January 2023 Publication History

Abstract

We study the robust stabilization problem of a class of nonlinear systems with asymmetric saturating actuators and mismatched disturbances. Initially, we convert such a robust stabilization problem into a nonlinear-constrained optimal control problem by constructing a discounted cost function for the auxiliary system. Then, for the purpose of solving the nonlinear-constrained optimal control problem, we develop a simultaneous policy iteration (PI) in the reinforcement learning framework. The implementation of the simultaneous PI relies on an actor–critic architecture, which employs actor and critic neural networks (NNs) to separately approximate the control policy and the value function. To determine the actor and critic NNs’ weights, we use the approach of weighted residuals together with the typical Monte-Carlo integration technique. Finally, we perform simulations of two nonlinear plants to validate the established theoretical claims.

References

[1]

Abu-Khalaf M., Huang J., Lewis F.L., Nonlinear H 2 / H ∞ constrained feedback control: A practical design approach using neural networks, Springer, London, 2006.

[2]

Abu-Khalaf M., Lewis F.L., Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach, Automatica 41 (2005) 779–791.

Digital Library

[3]

Basar T., Bernhard P., H ∞ optimal control and related minimax design problems: A dynamic game approach, 2nd ed., Birkhauser, Boston, MA, 1995.

[4]

Ben-Israel A., Greville T.N., Generalized inverses: Theory and applications, Springer Science & Business Media, New York, 2003.

[5]

Finlayson B., The method of weighted residuals and variational principles, Academic Press, New York, 1972.

[6]

Gao W., Jiang Z.-P., Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems, IEEE Transactions on Neural Networks and Learning Systems 29 (2018) 2614–2624.

[7]

Gao W., Jiang Y., Davari M., Data-driven cooperative output regulation of multi-agent systems via robust adaptive dynamic programming, IEEE Transactions on Circuits and Systems II: Express Briefs 66 (2019) 447–451.

[8]

Jiang Y., Jiang Z.P., Robust adaptive dynamic programming, John Wiley & Sons, Hoboken, New Jersey, 2017.

[9]

Jiang H., Zhang H., Luo Y., Cui X., H ∞ Control with constrained input for completely unknown nonlinear systems using data-driven reinforcement learning method, Neurocomputing 237 (2017) 226–234.

[10]

Khalil H.K., Nonlinear systems, 3rd ed., Prentice-Hall, Upper Saddle River, New Jersey, 2002.

[11]

Kiumarsi B., Vamvoudakis K.G., Modares H., Lewis F.L., Optimal and autonomous control using reinforcement learning: A survey, IEEE Transactions on Neural Networks and Learning Systems 29 (2018) 2042–2062.

[12]

Lewis F., Jagannathan S., Yesildirek A., Neural network control of robot manipulators and nonlinear systems, Taylor & Francis, London, 1999.

[13]

Lewis F.L., Vrabie D., Vamvoudakis K.G., Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers, IEEE Control Systems 32 (2012) 76–105.

[14]

Li J., Modares H., Chai T., Lewis F.L., Xie L., Off-policy reinforcement learning for synchronization in multiagent graphical games, IEEE Transactions on Neural Networks and Learning Systems 28 (2017) 2434–2445.

[15]

Liang M., Wang D., Liu D., Improved value iteration for neural-network-based stochastic optimal control design, Neural Networks 124 (2020) 280–295.

[16]

Lin F., Robust control design: An optimal control approach, Wiley-Interscience, Southern Gate, Chichester, 2007.

[17]

Littman M.L., Reinforcement learning improves behaviour from evaluative feedback, Nature 521 (2015) 445–451.

[18]

Liu D., Wei Q., Wang D., Yang X., Li H., Adaptive dynamic programming with applications in optimal control, Springer, Cham, Switzerland, 2017.

[19]

Liu D., Xue S., Zhao B., Luo B., Wei Q., Adaptive dynamic programming for control: A survey and recent advances, IEEE Transactions on Systems, Man, and Cybernetics: Systems 51 (2021) 142–160.

[20]

Liu D., Yang X., Wang D., Wei Q., Reinforecement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints, IEEE Transactions on Cybernetics 45 (2015) 1372–1385.

[21]

Luo B., Wu H.-N., Huang T., Off-policy reinforcement learning for H ∞ control design, IEEE Transactions on Cybernetics 45 (2015) 65–76.

[22]

Modares H., Lewis F.L., Jiang Z.-P., H ∞ Tracking control of completely unknown continuous-time systems via off-policy reinforcement learning, IEEE Transactions on Neural Networks and Learning Systems 26 (2015) 2550–2562.

[23]

Ren L., Zhang G., Mu C., Data-based H ∞ control for the constrained-input nonlinear systems and its applications in chaotic circuit systems, IEEE Transactions on Circuits and Systems. I. Regular Papers 67 (2020) 2791–2802.

[24]

Rudin W., Principles of mathematical analysis, McGraw-Hill Publishing Co, New York, 1976.

[25]

Rudin W., Functional analysis, McGraw-Hill Publishing Co, New York, 1991.

[26]

Vamvoudakis K.G., Miranda M.F., Hespanha J.P., Asymptotically stable adaptive-optimal control algorithm with saturating actuators and relaxed persistence of excitation, IEEE Transactions on Neural Networks and Learning Systems 27 (2016) 2386–2398.

[27]

Vrabie D., Lewis F., Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems, Neural Networks 22 (2009) 237–246.

Digital Library

[28]

Wang D., Liu D., Zhang Y., Li H., Neural network robust tracking control with adaptive critic framework for uncertain nonlinear systems, Neural Networks 97 (2018) 11–18.

[29]

Wang D., Mu C., Yang X., Liu D., Event-based constrained robust control of affine systems incorporating an adaptive critic mechanism, IEEE Transactions on Systems, Man, and Cybernetics: Systems 47 (2017) 1602–1612.

[30]

Wang D., Qiao J., Cheng L., An approximate neuro-optimal solution of discounted guaranteed cost control design, IEEE Transactions on Cybernetics 52 (2022) 77–86.

[31]

Wei Q., Li H., Yang X., He H., Continuous-time distributed policy iteration for multicontroller nonlinear systems, IEEE Transactions on Cybernetics 51 (2021) 2372–2383.

[32]

Wei Q., Song R., Yan P., Data-driven zero-sum neuro-optimal control for a class of continuous-time unknown nonlinear systems with disturbance using ADP, IEEE Transactions on Neural Networks and Learning Systems 27 (2016) 444–458.

[33]

Xue W., Fan J., Lopez V.G., Jiang Y., Chai T., Lewis F.L., Off-policy reinforcement learning for tracking in continuous-time systems on two time scales, IEEE Transactions on Neural Networks and Learning Systems 32 (2021) 4334–4346.

[34]

Xue S., Luo B., Liu D., Integral reinforcement learning based event-triggered control with input saturation, Neural Networks 131 (2020) 144–153.

[35]

Yang X., He H., Adaptive critic designs for optimal control of uncertain nonlinear systems with unmatched interconnections, Neural Networks 105 (2018) 142–153.

[36]

Yang X., He H., Self-learning robust optimal control for continuous-time nonlinear systems with mismatched disturbances, Neural Networks 99 (2018) 19–30.

[37]

Yang X., He H., Zhong X., Approximate dynamic programming for nonlinear-constrained optimizations, IEEE Transactions on Cybernetics 51 (2021) 2419–2432.

[38]

Yang X., Liu D., Ma H., Xu Y., Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems, Information Sciences 328 (2016) 435–454.

[39]

Yang X., Wei Q., An off-policy iteration algorithm for robust stabilization of constrained-input uncertain nonlinear systems, International Journal of Robust and Nonlinear Control 28 (2018) 5747–5765.

[40]

Yang X., Zhao B., Optimal neuro-control strategy for nonlinear systems with asymmetric input constraints, IEEE/CAA Journal of Automatica Sinica 7 (2020) 575–583.

[41]

Zhang H., Zhang K., Cai Y., Han J., Adaptive fuzzy fault-tolerant tracking control for partially unknown systems with actuator faults via integral reinforcement learning method, IEEE Transactions on Fuzzy Systems 27 (2019) 1986–1998.

[42]

Zhang H., Zhang K., Xiao G., Jiang H., Robust optimal control scheme for unknown constrained-input nonlinear systems via a plug-n-play event-sampled critic-only algorithm, IEEE Transactions on Systems, Man, and Cybernetics: Systems 50 (2020) 3169–3180.

[43]

Zhang S., Zhao B., Liu D., Zhang Y., Observer-based event-triggered control for zero-sum games of input constrained multi-player nonlinear systems, Neural Networks 144 (2021) 101–112.

[44]

Zhao B., Wang D., Shi G., Liu D., Li Y., Decentralized control for large-scale nonlinear systems with unknown mismatched interconnections via policy iteration, IEEE Transactions on Systems, Man, and Cybernetics: Systems 48 (2018) 1725–1735.

Cited By

Zhang DWang YJiang KLiang L(2024)Safe optimal robust control of nonlinear systems with asymmetric input constraints using reinforcement learningApplied Intelligence10.1007/s10489-023-05184-154:1(1-13)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-023-05184-1

Index Terms

Reinforcement learning for robust stabilization of nonlinear systems with asymmetric saturating actuators

Index terms have been assigned to the content through auto-classification.

Recommendations

Data-based robust adaptive control for a class of unknown nonlinear constrained-input systems via integral reinforcement learning

This paper presents a data-based robust adaptive control methodology for a class of nonlinear constrained-input systems with completely unknown dynamics. By introducing a value function for the nominal system, the robust control problem is transformed ...
Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming

This paper presents an adaptive dynamic programming-based guaranteed cost neural tracking control algorithm for a class of continuous-time matched uncertain nonlinear systems. By introducing an augmented system and employing a modified cost function ...
Output Feedback Stabilization for Time-Delay Nonlinear Interconnected Systems Using Neural Networks

In this paper, dynamic output feedback control problem is investigated for a class of nonlinear interconnected systems with time delays. Decentralized observer independent of the time delays is first designed. Then, we employ the bounds information of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neural Networks

Neural Networks Volume 158, Issue C

Jan 2023

386 pages

ISSN:0893-6080

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Science Ltd.

United Kingdom

Publication History

Published: 01 January 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang DWang YJiang KLiang L(2024)Safe optimal robust control of nonlinear systems with asymmetric input constraints using reinforcement learningApplied Intelligence10.1007/s10489-023-05184-154:1(1-13)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-023-05184-1

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents