Optimization [SMATB304]

Learning outcomes

This course is an introduction to the basic concepts of numerical optimization. Two questions are examined: how to characterize a solution and how to conceive a numerical method for finding it.

Objectives

The course focuses on the development and the study of numerical algorithms to solve unconstrained and constrained nonlinear optimization problems.

Content

Considering continuous but not necessarily convex unconstrained and constrained nonlinear optimization problems, the first and most important part of the course is devoted to the unconstrained case. After studying the characterization of minima for a generic unconstrained optimization problem (optimality conditions), we develop the main ideas behind the so-called line-search and trust-region approaches to globalize first- and second-order methods such as the steepest descent method, the Newton method and quasi-Newton methods, with some insight on both convergence and numerical considerations. The second part of the course is devoted to the constrained case and derives the Karush-Kuhn-Tucker optimality conditions, together with the key ideas of some well-known methods, among which the sequential quadratic programming method, the augmented Lagrangian method and the interior point method.

Table of contents

Introduction

Partie I : Unconstrained Optimization
  A. Optimality Conditions
  B. Overview of Algorithms
  C. Line Search Methods
  D. Trust-Region Methods
  E. Calculating Derivatives
  F. Least-Squares Problems
  G. Nonlinear Equations

Partie II : Constrained Optimization
  A. Optimality Conditions
  B. Overview of Algorithms

Exercises description

Exercise sessions are given for 1h30 per week.