Deep Learning Theory (CS 540).

Essential info.

Schedule.

Schedule will be continuously updated.

Notes URLs are always the new evolving version.

Date Topic Notes Tablet Assignments
8/23 Intro c1 lec1
8/25 Shallow approximation s2.1 lec2
8/30 Shallow approximation s2.2 lec3 hw1 tex, pdf.
9/1 Shallow approximation s2.3 lec4
9/6 Initialization and overparameterization s3.1 lec5
9/8 Initialization and overparameterization s3.1 lec6
9/13 Initialization and overparameterization s3.2 lec7
9/15 Initialization and overparameterization s3.2 lec8
9/20 Initialization and overparameterization s3.3 lec9
9/22 Architecture benefits s4.1 lec10
9/27 Architecture benefits lec11 hw1 due.
9/29 Optimization near initialization lec12
10/4 Optimization near initialization lec13
10/6 Optimization near initialization lec14
10/11 Optimization near initialization lec15
10/13 Optimization near initialization lec16
10/18 Clarke differentials lec17
10/20 Gradient flow and differential inclusions lec18 hw2 tex, pdf, py.
10/25 Positive homogeneity and norm preservation lec19
10/27 Positive homogeneity and norm preservation lec20
11/1 Margins lec21
11/3 Margins & project info lec22
11/8 No official lecture.
Optional lecture: transformers.
lec23
11/10 No class,
moved to 11/8.
11/15 Generalization lec24 hw2 due.
11/17
Generalization
lec25
project phase 1 due.
hw3 tex, pdf, py.
11/29 Generalization lec26
12/1 Generalization
12/6 Generalization
12/13
8-11am
(final exam slot)
Project poster session (online)
project phase 2 due.
12/15 hw3 due.

Homework policies.

Project information.

Details here.