1

Key concerns of numerical methods

No handouts

2

Performance: Arithmetic vs. memory

Performance experiments with matrix multiplication (PDF)

Ideal-cache terminology (PDF)

3

Memory optimization and cache obliviousness

Experiments with cache-oblivious matrix-multiplication (PDF)

4

Accuracy and floating-point arithmetic

Notes on floating-point (PDF)

*Note: Files in this section are from a previous version of the course.

5

Floating-point and numerical stability

No handouts

6

Backwards stability of summation, norms

No handouts

7

Condition numbers and eigenvalues

No handouts

8

The singular-value decomposition

No handouts

9

Least-square problems and QR factors

No handouts

10

Gram-Schmidt stability and householder QR

Gram-Schmidt notes (PDF)

Householder notes (PDF)

*Note: Files in this section are from a previous version of the course.

11

Householder, Gaussian, and Cholesky factorization

No handouts

12

Eigenproblems, characteristic polynomials, and Shur factors

No handouts

13

Hessenberg factorization and its applications, power methods

Hessenberg handout (PDF)

*Note: Files in this section are from a previous version of the course.

14

QR iteration for eigenproblems

No handouts

15

Overview of iterative and sparse solvers

No handouts

16

Arnoldi and Lanczos iterations

No handouts

17

Restarting Lanczos iterations

No handouts

18

GMRES and MINRES

No handouts

19

Steepest-descent and conjugate-gradient methods

Shewchuk, Jonathan Richard. "An Introduction to the Conjugate Gradient Method Without the Agonizing Pain." August 4, 1994. Pages 8, and 20. (PDF)

20

Preconditioning and condition numbers of PDE-like matrices

No handouts

21

Biconjugate-gradient methods, sparse-direct solvers

Summary of options for solving linear systems (PDF)

Notes on sparse-direct solvers (PDF)

*Note: Second file in this section is from a previous version of the course.

22

Nonlinear conjugate gradient, and conjugate-gradient eigensolvers

No handouts

23

Overview of optimization problems

Overview of optimization (PDF)

Notes on adjoint methods (PDF)

24

Adjoint methods and sensitivity analysis

No handouts

25

Adjoint methods for recurrences, CCSA algorithms

Adjoint methods for recurrence relations (PDF)

Svanberg, Krister. "A Class of Globally Convergent Optimization Methods Based on Conservative Convex Separable Approximations."SIAM Journal on Optimization 12, no. 2 (2002): 555-573. Pages 1-10.

26

Lagrange dual functions and KKT conditions

No handouts

27

Quasi-Newton methods

No handouts

28

BFGS updates

No handouts

29

Derivative-free optimization: Linear and quadratic models

No handouts

30

Global optimization and the DIRECT algorithm

Jones, D. R., C. D. Perttunen, and B. E. Stuckman. "Lipschitzian Optimization Without the Lipschitz Constance." Journal of Optimization Theory and Application 79 no. 1 (1993): 157. First few pages.(PDF - 1.5MB)

31

Numerical integration and accuracy of the trapezoidal rule

Notes on error analysis of the trapezoidal rule and Clenshaw-Curtis quadrature in terms of Fourier series. (PDF)

Two numerical experiments with trapezoidal rule (PDF)

32

Clenshaw-Curtis quadrature

"Clenshaw-Curtis Quadrature." Wikipedia.

33

Chebyshev approximation

No handouts

Problem set 1 (PDF)

matmul_bycolumn (M)

benchmul (M)

Problem set 1 solutions (PDF)

Problem set 2 (PDF)

loopsum (M)

div2sum (M)

Problem set 2 solutions (PDF)

Problem set 3 (PDF)

Problem set 3 solutions (PDF)

Problem set 4 (PDF)

Problem set 4 solutions (PDF)

Problem set 5 (PDF)

lanczos (M)

A363 (M)

SD (M)

A386 (M)

Problem set 5 solutions (PDF)

Fall 2010 midterm (PDF)

(PDF)

Fall 2009 midterm (PDF)

No solutions available

Fall 2008 midterm (PDF)

(PDF)