Optimization is one of the most important areas of modern applied mathematics, with applications in fields from engineering and economics to finance, statistics, management science, and medicine. While many books have addressed its various aspects, Nonlinear Optimization is the first comprehensive treatment that will allow graduate students and researchers to understand its modern ideas, principles, and methods within a reasonable time, but without sacrificing mathematical precision. Andrzej Ruszczynski, a leading expert in the optimization of nonlinear stochastic systems, integrates the theory and the methods of nonlinear optimization in a unified, clear, and mathematically rigorous fashion, with detailed and easy-to-follow proofs illustrated by numerous examples and figures. The book covers convex analysis, the theory of optimality conditions, duality theory, and numerical methods for solving unconstrained and constrained optimization problems. It addresses not only classical material but also modern topics such as optimality conditions and numerical methods for problems involving nondifferentiable functions, semidefinite programming, metric regularity and stability theory of set-constrained systems, and sensitivity analysis of optimization problems. Based on a decade's worth of notes the author compiled in successfully teaching the subject, this book will help readers to understand the mathematical foundations of the modern theory and methods of nonlinear optimization and to analyze new problems, develop optimality theory for them, and choose or construct numerical solution methods. It is a must for anyone seriously interested in optimization.
Cited By
- Mashwani W, Haider R, Brahim Belhaouari S and Khan A (2021). A Multiswarm Intelligence Algorithm for Expensive Bound Constrained Optimization Problems, Complexity, 2021, Online publication date: 1-Jan-2021.
- Liu C and Boumal N (2019). Simple Algorithms for Optimization on Riemannian Manifolds with Constraints, Applied Mathematics and Optimization, 82:3, (949-981), Online publication date: 1-Dec-2020.
- Drori Y and Taylor A (2019). Efficient first-order methods for convex minimization: a constructive approach, Mathematical Programming: Series A and B, 184:1-2, (183-220), Online publication date: 1-Nov-2020.
- Hahn G (2019). Optimal allocation of Monte Carlo simulations to multiple hypothesis tests, Statistics and Computing, 30:3, (571-586), Online publication date: 1-May-2020.
- Hang N, Mordukhovich B and Sarabi M (2018). Second-order variational analysis in second-order cone programming, Mathematical Programming: Series A and B, 180:1-2, (75-116), Online publication date: 1-Mar-2020.
- Zhang H (2019). New analysis of linear convergence of gradient-type methods via unifying error bound conditions, Mathematical Programming: Series A and B, 180:1-2, (371-416), Online publication date: 1-Mar-2020.
- Shao W and Zuo Y (2019). Computing the halfspace depth with multiple try algorithm and simulated annealing algorithm, Computational Statistics, 35:1, (203-226), Online publication date: 1-Mar-2020.
- Xue H, Wang L, Chen S and Wang Y (2019). A Primal Framework for Indefinite Kernel Learning, Neural Processing Letters, 50:1, (165-188), Online publication date: 1-Aug-2019.
- Boland N, Christiansen J, Dandurand B, Eberhard A and Oliveira F (2019). A parallelizable augmented Lagrangian method applied to large-scale non-convex-constrained optimization problems, Mathematical Programming: Series A and B, 175:1-2, (503-536), Online publication date: 1-May-2019.
- Liu S (2019). A simple version of bundle method with linear programming, Computational Optimization and Applications, 72:2, (391-412), Online publication date: 1-Mar-2019.
- Kanatsoulis C, Fu X, Sidiropoulos N and Hong M (2018). Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data, IEEE Transactions on Signal Processing, 67:2, (306-319), Online publication date: 15-Jan-2019.
- Zhao N, Liu R, Chen Y, Wu M, Jiang Y, Xiong W and Liu C (2018). Contract design for relay incentive mechanism under dual asymmetric information in cooperative networks, Wireless Networks, 24:8, (3029-3044), Online publication date: 1-Nov-2018.
- Duarte B, Sagnol G and Wong W (2018). An algorithm based on semidefinite programming for finding minimax optimal designs, Computational Statistics & Data Analysis, 119:C, (99-117), Online publication date: 1-Mar-2018.
- Du Y and Ruszczyński A (2017). Rate of Convergence of the Bundle Method, Journal of Optimization Theory and Applications, 173:3, (908-922), Online publication date: 1-Jun-2017.
- Bandeira A, Boumal N and Singer A (2017). Tightness of the maximum likelihood semidefinite relaxation for angular synchronization, Mathematical Programming: Series A and B, 163:1-2, (145-167), Online publication date: 1-May-2017.
- Curtis F, Robinson D and Samadi M (2017). A trust region algorithm with a worst-case iteration complexity of $$\mathcal{O}(\epsilon ^{-3/2})$$O(∈-3/2) for nonconvex optimization, Mathematical Programming: Series A and B, 162:1-2, (1-32), Online publication date: 1-Mar-2017.
- Muoi B and Yen N (2017). Local Stability and Local Convergence of the Basic Trust-Region Method, Journal of Optimization Theory and Applications, 172:2, (578-593), Online publication date: 1-Feb-2017.
- Khanh P, Yao J and Yen N (2017). The Mordukhovich Subdifferentials and Directions of Descent, Journal of Optimization Theory and Applications, 172:2, (518-534), Online publication date: 1-Feb-2017.
- Boumal N, Voroninski V and Bandeira A The non-convex Burer–Monteiro approach works on smooth semidefinite programs Proceedings of the 30th International Conference on Neural Information Processing Systems, (2765-2773)
- Dentcheva D, Martinez G and Wolfhagen E (2016). Augmented Lagrangian Methods for Solving Optimization Problems with Stochastic-Order Constraints, Operations Research, 64:6, (1451-1465), Online publication date: 1-Dec-2016.
- (2016). Initialization-free distributed algorithms for optimal resource allocation with feasibility constraints and application to economic dispatch of power systems, Automatica (Journal of IFAC), 74:C, (259-269), Online publication date: 1-Dec-2016.
- St.Amand J and Huan J Discriminative View Learning for Single View Co-Training Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, (2221-2226)
- Das H (2016). Decomposition Procedure for Solving NLP and QP Problems based on Lagrange and Sander's Method, International Journal of Operations Research and Information Systems, 7:4, (67-93), Online publication date: 1-Oct-2016.
- Kantaros Y and Zavlanos M (2016). Global Planning for Multi-Robot Communication Networks in Complex Environments, IEEE Transactions on Robotics, 32:5, (1045-1061), Online publication date: 1-Oct-2016.
- Morgan D, Subramanian G, Chung S and Hadaegh F (2016). Swarm assignment and trajectory optimization using variable-swarm, distributed auction assignment and sequential convex programming, International Journal of Robotics Research, 35:10, (1261-1285), Online publication date: 1-Sep-2016.
- Dao M, Gwinner J, Noll D and Ovcharova N (2016). Nonconvex bundle method with application to a delamination problem, Computational Optimization and Applications, 65:1, (173-203), Online publication date: 1-Sep-2016.
- Byrd R, Chin G, Nocedal J and Oztoprak F (2016). A family of second-order methods for convex $$\ell _1$$ℓ1-regularized optimization, Mathematical Programming: Series A and B, 159:1-2, (435-467), Online publication date: 1-Sep-2016.
- Li C, Gao X, Li Y and Liu R (2016). A new neural network for l1-norm programming, Neurocomputing, 202:C, (98-103), Online publication date: 19-Aug-2016.
- Sikora T and Magoulas G (2016). Evolutionary approaches to signal decomposition in an application service management system, Soft Computing - A Fusion of Foundations, Methodologies and Applications, 20:8, (3063-3084), Online publication date: 1-Aug-2016.
- Wiese T, Riemensberger M and Utschick W (2016). Scheduling for Network-Coded Multicast With Interference, IEEE Transactions on Signal Processing, 64:9, (2245-2254), Online publication date: 1-May-2016.
- Kinsner W and Van Dang H (2016). Multiobjective Multivariate Optimization of Joint Spectrum Sensing and Power Control in Cognitive Wireless Networks, International Journal of Cognitive Informatics and Natural Intelligence, 10:2, (26-51), Online publication date: 1-Apr-2016.
- Kantaros Y and Zavlanos M (2016). Distributed communication-aware coverage control by mobile sensor networks, Automatica (Journal of IFAC), 63:C, (209-220), Online publication date: 1-Jan-2016.
- Wang J and Ye J Multi-layer feature reduction for tree structured group lasso via hierarchical projection Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1, (1279-1287)
- Liu Yang , Liping Jing and Ng M (2015). Robust and Non-Negative Collective Matrix Factorization for Text-to-Image Transfer Learning, IEEE Transactions on Image Processing, 24:12, (4701-4714), Online publication date: 1-Dec-2015.
- Yu B, Liu D, Chowdhury S and Pan D TILA Proceedings of the IEEE/ACM International Conference on Computer-Aided Design, (110-117)
- Drusvyatskiy D and Ioffe A (2015). Quadratic growth and critical point stability of semi-algebraic functions, Mathematical Programming: Series A and B, 153:2, (635-653), Online publication date: 1-Nov-2015.
- Asamov T and Ruszczyński A (2015). Time-consistent approximations of risk-averse multistage stochastic optimization problems, Mathematical Programming: Series A and B, 153:2, (459-493), Online publication date: 1-Nov-2015.
- Lorenz D, Pfetsch M and Tillmann A (2015). Solving Basis Pursuit, ACM Transactions on Mathematical Software, 41:2, (1-29), Online publication date: 4-Feb-2015.
- Lan G (2015). Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization, Mathematical Programming: Series A and B, 149:1-2, (1-45), Online publication date: 1-Feb-2015.
- Wang J, Wonka P and Ye J (2015). Lasso screening rules via dual polytope projection, The Journal of Machine Learning Research, 16:1, (1063-1101), Online publication date: 1-Jan-2015.
- Zhao N, Wu M, Xiong W and Liu C (2015). Cooperative communication in cognitive radio networks under asymmetric information, International Journal of Distributed Sensor Networks, 2015, (2-2), Online publication date: 1-Jan-2015.
- Wang J and Ye J Two-layer feature reduction for Sparse-Group Lasso via decomposition of convex sets Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, (2132-2140)
- Chi E, Zhou H and Lange K (2014). Distance majorization and its applications, Mathematical Programming: Series A and B, 146:1-2, (409-436), Online publication date: 1-Aug-2014.
- Herman G Superiorization for Image Analysis Proceedings of the 16th International Workshop on Combinatorial Image Analysis - Volume 8466, (1-7)
- Sensoy M, Fokoue A, Pan J, Norman T, Tang Y, Oren N and Sycara K Reasoning about uncertain information and conflict resolution through trust revision Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems, (837-844)
- Zhang D and Lu Z (2013). Assessing the Value of Dynamic Pricing in Network Revenue Management, INFORMS Journal on Computing, 25:1, (102-115), Online publication date: 1-Jan-2013.
- Topaloglu H, Birbil S, Frenk J and Noyan N (2012). Tractable Open Loop Policies for Joint Overbooking and Capacity Control Over a Single Flight Leg with Multiple Fare Classes, Transportation Science, 46:4, (460-481), Online publication date: 1-Nov-2012.
- Lu Z and Zhang Y (2012). An augmented Lagrangian approach for sparse principal component analysis, Mathematical Programming: Series A and B, 135:1-2, (149-193), Online publication date: 1-Oct-2012.
- de la Puente P and Censi A Dense map inference with user-defined priors Proceedings of the 2012 international conference on Spatial Cognition VIII, (94-113)
- Rusmevichientong P and Topaloglu H (2012). Robust Assortment Optimization in Revenue Management Under the Multinomial Logit Choice Model, Operations Research, 60:4, (865-882), Online publication date: 1-Jul-2012.
- Molyboha A and Zabarankin M (2012). Stochastic Optimization of Sensor Placement for Diver Detection, Operations Research, 60:2, (292-312), Online publication date: 1-Mar-2012.
- Waeber R, Frazier P and Henderson S A Bayesian approach to stochastic root finding Proceedings of the Winter Simulation Conference, (4038-4050)
- Sabo K, Scitovski R and Vazler I (2011). Searching for a Best Least Absolute Deviations Solution of an Overdetermined System of Linear Equations Motivated by Searching for a Best Least Absolute Deviations Hyperplane on the Basis of Given Data, Journal of Optimization Theory and Applications, 149:2, (293-314), Online publication date: 1-May-2011.
- Angelosante D, Giannakis G and Sidiropoulos N (2010). Estimating multiple frequency-hopping signal parameters via sparse linear regression, IEEE Transactions on Signal Processing, 58:10, (5044-5056), Online publication date: 1-Oct-2010.
- Mc Clary D, Syrotiuk V and Kulahci M (2010). Profile-driven regression for modeling and runtime optimization of mobile networks, ACM Transactions on Modeling and Computer Simulation, 20:3, (1-24), Online publication date: 1-Sep-2010.
- Akimoto Y, Nagata Y, Ono I and Kobayashi S Theoretical analysis of evolutionary computation on continuously differentiable functions Proceedings of the 12th annual conference on Genetic and evolutionary computation, (1401-1408)
- Sivaramakrishnan K (2010). A parallel interior point decomposition algorithm for block angular semidefinite programs, Computational Optimization and Applications, 46:1, (1-29), Online publication date: 1-May-2010.
- Lin H and Yeh J (2009). Optimal reduction of solutions for support vector machines, Applied Mathematics and Computation, 214:2, (329-335), Online publication date: 1-Aug-2009.
- Saha B and Ray N (2009). Image thresholding by variational minimax optimization, Pattern Recognition, 42:5, (843-856), Online publication date: 1-May-2009.
- Gorinevsky D, Kim S, Beard S, Boyd S and Gordon G (2009). Optimal estimation of deterioration from diagnostic image sequence, IEEE Transactions on Signal Processing, 57:3, (1030-1043), Online publication date: 1-Mar-2009.
- Tang J, Pan Z, Fung R and Lau H (2009). Vehicle routing problem with fuzzy time windows, Fuzzy Sets and Systems, 160:5, (683-695), Online publication date: 1-Mar-2009.
- Dahl G and Flatberg T (2009). Reconstructing (0,1)-matrices from projections using integer programming, Computational Optimization and Applications, 42:1, (141-154), Online publication date: 1-Jan-2009.
- Luo P, Zhuang F, Xiong H, Xiong Y and He Q Transfer learning from multiple source domains via consensus regularization Proceedings of the 17th ACM conference on Information and knowledge management, (103-112)
- Johnson M and Farid H Exposing digital forgeries through specular highlights on the eye Proceedings of the 9th international conference on Information hiding, (311-325)
- Yu B, Liu D, Chowdhury S and Pan D TILA: Timing-driven incremental layer assignment 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), (110-117)
- Shi Q, Hong M, Song E, Cai Y and Xu W A penalty-BSUM approach for rate optimization in full-duplex MIMO relay networks with relay processing delay 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (3646-3650)
Index Terms
- Nonlinear Optimization
Recommendations
A Projected Lagrangian Algorithm for Nonlinear Minimax Optimization
The minimax problem is an unconstrained optimization problem whose objective function is not differentiable everywhere, and hence cannot be solved efficiently by standard techniques for unconstrained optimization. It is well known that the problem can ...
Nonlinear complementarity as unconstrained optimization
Several methods for solving the nonlinear complementarity problem (NCP) are developed. These methods are generalizations of the recently proposed algorithms of Mangasarian and Solodov (Ref. 1) and are based on an unconstrianed minimization formulation ...