Optimization: Unconstrained nonlinear optimization

Introduction to unconstrained nonlinear optimization, Newton’s algorithms and descent methods.

Description

Introduction to unconstrained nonlinear optimization, Newton’s algorithms and descent methods.

What you will learn

• Formulation: you will learn from simple examples how to formulate, transform and characterize an optimization problem.


• Objective function: you will review the mathematical properties of the objective function that are important in optimization.


• Optimality conditions: you will learn sufficient and necessary conditions for an optimal solution.

Solving equations, Newton: this is a reminder about Newton's method to solve nonlinear equations.

• Newton's local method: you will see how to interpret and adapt Newton's method in the context of optimization.


• Descent methods: you will learn the family of descent methods, and its connection with Newton's method.

Prerequisites

The course assumes no prior knowledge of optimization. It relies heavily on linear algebra, analysis and calculus (matrices, derivatives, eigenvalues, etc.)
The knowledge of the programming language Python is an asset to learn the details of the algorithms. However, it is possible to follow the course without programming at all.

Plan

The course is structured into 6 sections. 
Formulation: you will learn from simple examples how to formulate, transform and characterize  an optimization problem.  

• Objective function: you will review the mathematical properties of the objective function that are important in optimization.


• Optimality conditions: you will learn sufficient and necessary conditions for an optimal solution. 


• Solving equations, Newton: this is a reminder about Newton's method to solve nonlinear equations.


• Newton's local method: you will see how to interpret and adapt Newton's method in the context of optimization.


• Descent methods: you will learn the family of descent methods, and its connection with Newton's method.