Logistics.
Course staff. (Contact us via private edstem posts.)
Instructor: Matus Telgarsky.
TAs: Justin Li, Danny Son.
Evaluation.
80% is across 4 homeworks. (See below for policies.)
20% is a paper reading or an open problem project.
All work must be typed (however you wish, but typed), and submitted at gradescope.
No late work.
Do not cheat in this class, just drop it and take an easier one. No one cares whether you did or did not take this class.
Readings and prerequisites.
Basic math and proof writing; use the lecture notes and hw1 (out 8/31) to help determine readiness.
Course lecture notes: new evolving version or old version.
Basic machine learning, see for instance my 446 materials.
“Understanding Machine Learning”, by Shai Shalev-Shwartz and Shai Ben-David, can be downloaded from that page, is free for personal use. I think this book is a wonderful resource, I find its presentation very clear, direct, and minimal.
Schedule will be continuously updated.
Notes URLs are always the new evolving version.
| Date | Topic | Notes | Tablet | Assignments |
|---|---|---|---|---|
| 8/23 | Intro | c1 | lec1 | |
| 8/25 | Shallow approximation | s2.1 | lec2 | |
| 8/30 | Shallow approximation | s2.2 | lec3 | hw1 tex, pdf. |
| 9/1 | Shallow approximation | s2.3 | lec4 | |
| 9/6 | Initialization and overparameterization | s3.1 | lec5 | |
| 9/8 | Initialization and overparameterization | s3.1 | lec6 | |
| 9/13 | Initialization and overparameterization | s3.2 | lec7 | |
| 9/15 | Initialization and overparameterization | s3.2 | lec8 | |
| 9/20 | Initialization and overparameterization | s3.3 | lec9 | |
| 9/22 | Architecture benefits | s4.1 | lec10 | |
| 9/27 | Architecture benefits | lec11 | hw1 due. | |
| 9/29 | Optimization near initialization | lec12 | ||
| 10/4 | Optimization near initialization | lec13 | ||
| 10/6 | Optimization near initialization | lec14 | ||
| 10/11 | Optimization near initialization | lec15 | ||
| 10/13 | Optimization near initialization | lec16 | ||
| 10/18 | Clarke differentials | lec17 | ||
| 10/20 | Gradient flow and differential inclusions | lec18 | hw2 tex, pdf, py. | |
| 10/25 | Positive homogeneity and norm preservation | lec19 | ||
| 10/27 | Positive homogeneity and norm preservation | lec20 | ||
| 11/1 | Margins | lec21 | ||
| 11/3 | Margins & project info | lec22 | ||
| 11/8 | No official
lecture. Optional lecture: transformers. |
lec23 | ||
| 11/10 | No class, moved to 11/8. |
|||
| 11/15 | Generalization | lec24 | hw2 due. | |
| 11/17 |
Generalization |
lec25 |
project phase 1 due. hw3 tex, pdf, py. |
|
| 11/29 | Generalization | lec26 | ||
| 12/1 | Generalization | lec27 | ||
| 12/6 | Generalization | |||
| 12/13 8-11am |
(final exam slot) Project poster session (online) |
project phase 2 due. | ||
| 12/15 | hw3 due. |
Four homeworks, each 20% of course grade.
Homework must be typeset (albeit however you wish: latex, markdown, etc), and submitted in gradescope.
No late homework.
Academic integrity.
All submitted homework must be in your own words; keep your discussions sufficiently high level to prevent claims of academic integrity violations.
You may discuss in more detail with at most three other students, list their ID numbers on the first page of the homework.
I don’t expect you to be able to find homework solutions online (I don’t use online resources to come up with problems); if you do rely upon external resources, cite them properly, and still write your solutions in your own words. Please see Jeff Erickson’s discussion of academic integrity.
When integrity violations are found, they will be submitted to the department’s evaluation board.