This comprehensive textbook covers both classical and geometric aspects of optimization using methods, deterministic and stochastic, in a single volume and in a language accessible to non-mathematicians. It will help serve as an ideal study material for senior undergraduate and graduate students in the fields of civil, mechanical, aerospace, electrical, electronics, and communication engineering.
The book includes:
- Derivative-based Methods of Optimization.
- Direct Search Methods of Optimization.
- Basics of Riemannian Differential Geometry.
- Geometric Methods of Optimization using Riemannian Langevin Dynamics.
- Stochastic Analysis on Manifolds and Geometric Optimization Methods.
This textbook comprehensively treats both classical and geometric optimization methods, including deterministic and stochastic (Monte Carlo) schemes. It offers an extensive coverage of important topics including derivative-based methods, penalty function methods, method of gradient projection, evolutionary methods, geometric search using Riemannian Langevin dynamics and stochastic dynamics on manifolds. The textbook is accompanied by online resources including MATLAB codes which are uploaded on our website. The textbook is primarily written for senior undergraduate and graduate students in all applied science and engineering disciplines and can be used as a main or supplementary text for courses on classical and geometric optimization.
Chapter 1 Optimization methods – A preview
1.2 The continuous case – mathematical formulation
1.3 The discrete case – The travelling salesman problem
1.4 Basics of probability theory and random number generation
1.5 The brachistochrone problem
1.6 More on functional optimization: Hamilton’s principle
1.7 Constrained optimization problems and optimality conditions
1.8. Functional optimization and optimal control
Chapter 2 Classical derivative-based methods of optimization
2.2 Basic gradient methods
2.3 Quasi-Newton methods
2.4 Penalty function methods
2.5 Linear programming (LP)
2.6. Method of generalized reduced gradients
2.7 Method of feasible directions
2.8 Method of gradient projection
Chapter 3 – Classical derivative-free methods of optimization
3.2 Direct search methods
3.3 Other direct search methods
3.4 Metaheuristics - Evolutionary methods
Chapter 4 Elements of Riemannian Differential Geometry and geometric methods of optimization
4.2 Tangent vectors and tangent space on manifolds
4.3 Riemannian (geometric) version of some classical gradient methods
4.4. Statistical estimation by geometrical method of optimization
4.5. Stochastic processes, stochastic calculus and solution of SDEs
4.6. Analogy between statistical sampling and stochastic optimization
4.7. Geometric method of optimization by Riemannian Langevin dynamics
Chapter 5 Stochastic analysis on a manifold and more on geometric optimization methods
5.2 Stochastic development on a manifold
5.3. Non-convex function optimization based on stochastic development
5.4. Parameter estimation by GALA