Machine Learning CSCI 567, Fall 2023

Instructor: Dani Yogatama (yogatama@usc.edu)
Teaching assistants: Ting-Rui Chiang, Robby Costales, Samuel Griesemer, Tenghao Huang, Oliver Liu, Joshua Robinson, Tejas Srinivasan
Course producers: Sneha Bandi, Aman Bansal, Sanying Yi, Wenda Zhou
When: Fridays, Lecture: 1:00-3:20pm, Discussion: 3:30-4:20pm
Where: SGM 123

Objective

The major objective of this course is to introduce modern machine learning methods that are commonly used in the real world to build state-of-the-art systems. Particular focus will be laid on the conceptual understanding of these techniques, their applications, and hands-on experience.

Prerequisites

  1. Undergraduate level training or coursework in linear algebra, multivariate calculus, basic probability and statistics;
  2. Familiarity with basic machine learning concepts from taking an introductory machine learning class or similar experience,
  3. Skills in programming with Python (self-studying numpy, pytorch/jax, scipy, and scikit- learn and related packages is expected).

Grading

Exercises

3 programming assignments will be released as take-home exercises. These will not be graded and are for your own practice.

Discussions

Attending the discussion sessions is required. The discussion provides more detailed and in- depth exposition of the lecture materials as well as useful hints for take-home exercises.

Syllabus

Topics Notes
8/25 Course overview, core ML concepts (supervised vs. unsupervised learning, convex vs. non convex optimization) [lecture 1 slides]
(Reference: Part 1 of Deep Learning)
9/1 Linear regression, overfitting and regularization [lecture 2, 3 slides]
(Reference: Stanford CS 229 Lecture Note)
9/8 Linear classifier, surrogate losses, gradient descent, perceptron, logistic regression (Discussion: Final Project Expectation)
9/15 Multiclass classification, feed forward neural networks, convolutional neural network [lecture 4 slides]
Project proposal deadline
Programming assignment 1
(Discussion: PyTorch Basics/ MNIST Example )
9/22 Recurrent neural networks, attention, transformers, memory networks [lecture 5 slides]
(Discussion: Transformers)
9/29 Quiz 1 [Quiz 1]
10/6 Large language models, pretraining, instruction tuning, sampling, prompting [lecture 6 slides]
Programming assignment 2
10/13 Fall recess
10/20 Latent variable models, expectation maximization, variational autoencoders [lecture 8 slides/EM Demo]
(Discussion: VQ-VAE / Diffusion )
10/27 Density estimation, Naive Bayes, Principal Component Analysis (PCA) [lecture 9 slides]
(Discussion: Reinforcement Learning)
11/3 Safety, biases, fairness [lecture 10 slides]
Programming assignment 3
11/10 Veterans Day
11/17 Quiz 2
11/24 Thanksgiving
12/1 Poster session Final project report due 12/8

Academic Policies

Students with disabilities: Any student requesting academic accommodations based on a disability is required to register with Disability Services and Programs (DSP) each semester. A letter of verification for approved accommodations can be obtained from DSP. Please be sure the letter is delivered to the instructor or to a TA as early in the semester as possible.

Academic integrity: USC seeks to maintain an optimal learning environment. General principles of academic honesty include the concept of respect for the intellectual property of others, the expectation that individual work will be submitted unless otherwise allowed by an instructor, and the obligations both to protect one's own academic work from misuse by others as well as to avoid using another's work as one's own. All students are expected to understand and abide by these principles. Scampus, the Student Guidebook, contains the Student Conduct Code in Section 11.00. Students will be referred to the Office of Student Judicial Affairs and Community Standards for further review, should there be any suspicion of academic dishonesty.