Machine Learning CSCI 567, Fall 2024

Instructor: Dani Yogatama (yogatama@usc.edu). Office hours: Thursdays 9am PT
Teaching assistants: Yavuz Faruk Bakman, Robby Costales, Xiao Fu, Isabelle Lee, Sajjad Shahabi, Xinyan Yu
Course producers: TBD
When: Fridays, Lecture: 2:00-4:20pm, Discussion: 4:30-5:20pm
Where: TBD

Objective

The major objective of this course is to introduce modern machine learning methods that are commonly used in the real world to build state-of-the-art systems. Particular focus will be laid on the conceptual understanding of these techniques, their applications, and hands-on experience.

Prerequisites

  1. Undergraduate level training or coursework in linear algebra, multivariate calculus, basic probability and statistics;
  2. Familiarity with basic machine learning concepts from taking an introductory machine learning class or similar experience,
  3. Skills in programming with Python (self-studying numpy, pytorch/jax, scipy, and scikit- learn and related packages is expected).

Grading

Exercises

3 programming assignments will be released as take-home exercises. These will not be graded and are for your own practice.

Discussions

Attending the discussion sessions is required. The discussion provides more detailed and in- depth exposition of the lecture materials as well as useful hints for take-home exercises.

Syllabus

Topics Notes
8/30 Course overview [lecture 1 slides]
[math review]
(Reference: Part 1 of Deep Learning)
9/6 Linear regression, nonlinear regression, overfitting and regularization [lecture 2 slides]
9/13 Gradient descent, linear classifier, surrogate losses, perceptron, logistic regression [lecture 3 slides]
[programming assignment 1]
9/20 Multiclass classification, feed forward neural networks, convolutional neural network [lecture 4 slides part 1]
[lecture 4 slides part 2]
9/27 Transformers, attention [slides by Lucas Beyer]
10/4 Quiz 1
10/11 Fall Recess
10/18 Large language models, pretraining, instruction tuning, sampling, prompting [slides by Jesse Mu] [programming assignment 2]
10/25 Clustering, Gaussian Mixture Model [lecture 7 slides]
[EM demo]
11/1 Density estimation, Naive Bayes, Principal Component Analysis (PCA) [lecture 8 slides]
11/8 Decision trees, boosting [lecture 9 slides]
11/15 Multi armed bandits, reinforcement learning [lecture 10 slides]
11/22 Responsible ML [lecture 11 slides]
[programming assignment 3]
11/29 Thanksgiving break
12/6 Quiz 2 Final project report due 12/13

Academic Policies

Students with disabilities: Any student requesting academic accommodations based on a disability is required to register with Disability Services and Programs (DSP) each semester. A letter of verification for approved accommodations can be obtained from DSP. Please be sure the letter is delivered to the instructor or to a TA as early in the semester as possible.

Academic integrity: USC seeks to maintain an optimal learning environment. General principles of academic honesty include the concept of respect for the intellectual property of others, the expectation that individual work will be submitted unless otherwise allowed by an instructor, and the obligations both to protect one's own academic work from misuse by others as well as to avoid using another's work as one's own. All students are expected to understand and abide by these principles. Scampus, the Student Guidebook, contains the Student Conduct Code in Section 11.00. Students will be referred to the Office of Student Judicial Affairs and Community Standards for further review, should there be any suspicion of academic dishonesty.