| Instructor | Kuan Yang |
|---|---|
| Lecture times | Monday 18:00 - 19:40 (Week 1 - 16) Thursday 08:00 - 09:40 (Even weeks) |
| Location | Middle Hall 411 (中院 411) |
| Teaching assistant(s) | Guoliang Qiu (邱国良) |
| Office hours | Wednesday 14:00 - 15:00 at Room 1402, School of Software |
12/28 Homework 9 announced, due: Jan. 04.12/17 Homework 8 announced, due: Dec. 28.12/07 Homework 7 announced, due: Dec. 17.11/19 Homework 6 announced, due: Nov. 30.11/02 Homework 5 announced, due: Nov. 09.10/22 Homework 4 announced, due: Nov. 02.10/15 Homework 3 announced, due: Oct. 22.10/08 Homework 2 announced, due: Oct. 15.09/22 Homework 1 announced, due: Sept. 30.
Convex Optimization by Stephen Boyd and Lieven Vandenberghe, Cambridge University Press
An Introduction to Optimization by Edwin K. P. Chong and Stanislaw H. Żak, Wiley.
(中文翻译版：最优化导论（第四版），电子工业出版社)
Large-Scale Optimization for Data Science at Princeton
Convex Optimization at CMU
| Week | Date | Topics | Materials | Homework |
|---|---|---|---|---|
| 1 | 09/13 (Mon.) | Introduction to the course | Lec.1 manuscript | |
| 09/18 (W2 Mon.) | Analysis in vector spaces (I): compact sets, continuity, differential | Lec.2 manuscript | HW1 | |
| 2 | 09/23 (Thurs.) | Analysis in vector spaces (II): Hessian, definite matrices | Lec.3 manuscript | |
| 3 | 09/27 (Mon.) | Geometry: affine and convex sets, convex hull, polytope, simplex | Lec.4 manuscript | HW2 |
| 4 | 10/09 (W4 Thurs.) | Separating hyperplane theorem, supporting hyperplane theorem | Lec.5 manuscript | |
| 5 | 10/11 (Mon.) | Convex functions, midpoint convexity | Lec.6 manuscript | HW3 |
| 6 | 10/18 (Mon.) | First and second order conditions, convexity-preserving operations | Lec.7 manuscript | |
| 10/21 (Thurs.) | Convexity-preserving operations, inequalities, optimization problems | Lec.8 manuscript | HW4 | |
| 7 | 10/25 (Mon.) | Linear programming, simplex method | Lec.9 manuscript | |
| 8 | 11/01 (Mon.) | Farkas' lemma, duality of LP | Lec.10 manuscript | HW5 |
| 11/04 (Thurs.) | Applications of LP duality, unconstrained optimization, introduction to gradient descent | Lec.11 manuscript | ||
| 9 | 11/08 (Mon.) | Convergence of gradient descent, discrete-time Lyapunov's stability theorem, -smooth functions, convergence rate for fixed step size | Lec.12 manuscript | |
| 10 | 11/15 (Mon.) | -strongly convex functions, exponential convergence rate, exact line search method, Newton's method for finding roots | Lec.13 manuscript | |
| 11/18 (Thurs.) | Backtracking line search method, Newton's method for optimization | Lec.14 manuscript | HW6 | |
| 11 | 11/22 (Mon.) | Newton's method (cont'd), Proximal gradient descent | Lec.15 manuscript | |
| 12 | 11/29 (Mon.) | Properties of proximal mapping, equality constrained optimization, optimality condition for linear constraints, submanifolds | Lec.16 manuscript | |
| 12/02 (Thurs.) | Implicit function theorem, tangent space, differential on submanifolds, Lagrange multiplier | Lec.17 manuscript | ||
| 13 | 12/06 (Mon.) | Lagrange multiplier (cont'd), Second-order condition, Solving quadratic optimization | Lec.18 manuscript | HW7 |
| 14 | 12/13 (Mon.) | Newton's method, KKT condition | Lec.19 manuscript | |
| 12/16 (Thurs.) | KKT condition (cont'd), Lagrange dual | Lec.20 manuscript | HW8 | |
| 15 | 12/20 (Mon.) | Weak and strong duality, Slater's condition for convex problems | Lec.21 manuscript | |
| 16 | 12/27 (Mon.) | Projected gradient descent, introduction to interior-point methods | Lec.22 manuscript | HW9 |
| 12/30 (Thurs.) | Review and summary | Course summary |