Quadratic forms and matrix factorisation. Convexity. The theory of optimization with and without constraints: Lagrange functions, Kuhn-Tucker theory. Duality. Methods for optimization without constraints: line search, steepest descent, Newton methods, conjugate directions, non-linear least squares optimization. Methods for optimization with constraints: linear optimization, the simplex method, quadratic programming, penalty and barrier methods.