Course Information
- Instructor: Prof. Jin Sun
- Time: Mon/Wed 13:50 PM - 15:50 PM
- Location: Building 1002 Geography and Geology, Room 0143
- Office Hours: Wed 4:00 PM - 5:00 PM or by appointment
Description
Advanced Representation Learning is a course designed to delve deeper into the fundamental concepts of representation learning and its applications. In this class, students will explore various representation learning techniques, including both classical and deep learning methods, and learn how to apply these techniques to solve complex problems in computer vision, natural language processing, audio, and other areas. By working on the research project component of the course, the students will develop novel methods and theories about representation learning and prepare manuscripts describing their findings. By the end of this course, the students will have a solid understanding of the state-of-the-art in representation learning and be able to apply these techniques to solve real-world problems.Learning Outcomes
- Demonstrate understanding of machine learning and deep neural network fundamentals.
- Gain experience deploying deep learning models in computer vision, natural language processing, and audio domains.
(FREE) Textbooks
-
Deep Learning
by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
http://www.deeplearningbook.org/ -
Dive into Deep Learning
by Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola
https://d2l.ai/ -
Computer Vision: Algorithms and Applications
by Richard Szeliski
https://szeliski.org/Book/ - Machine Learning: a Probabilistic Perspective by Kevin Murphy
-
Foundations of Data Science
by Avrim Blum, John Hopcroft, and Ravindran Kannan
https://www.cs.cornell.edu/jeh/book.pdf
Course Schedule
Introduction
Week 1 Aug 13
Topic: Introduction and overview
Week 1 Aug 18
Topic: Data and dimensionality
Week 2 Aug 20
Topic: Dimension reduction, metric learning, PCA, MDS
Week 2 Aug 25
Topic: Structures in data spaces, manifolds, subspaces, sparse coding
Vision
Week 3 Aug 27
Topic: Pixels, 3D points, and cameras
Week 3 Sep 1
Topic: Labor day
Week 3 Sep 3
Topic: Image operations and image semantics
Week 4 Sep 8
Topic: Videos
Week 4 Sep 10
Topic: Image subspaces and manipulations
Language
Week 5 Sep 15
Topic: Representing words and sentences
Week 5 Sep 17
Topic: Language model pretraining
Week 6 Sep 22
Topic: NLP tasks
Week 6 Sep 24
Topic: Zero-shot and in-context learning
Audio
Week 7 Sep 29
Topic: Representing sound
Week 7 Oct 1
Topic: Audio generation and editing
Midterm Oct 6
Graphs
Week 8 Oct 8
Topic: Graphs and neural networks
Week 9 Oct 13
Topic: GNN applications
Multi-modal
Week 9 Oct 15
Topic: Overview of multi-modal learning
Week 10 Oct 20
Topic: Multimodal representation alignment
Advanced topics
Week 10 Oct 22
Topic: Unsupervised learning
Week 11 Oct 27
Topic: Self-supervised learning
Week 11 Oct 29
Topic: Domain adaptation and transfer learning
Week 12 Nov 3
Topic: LLM finetuning
Week 12 Nov 5
Topic: Generative models
Week 12 Nov 10
Topic: AI agent
Week 12 Nov 12
Topic: Time series
Week 13 Nov 17
Topic: Understanding LLM, VLM, etc
Week 13 Nov 19
Topic: Class review and discussion
Project
Project Presentation 1 Nov 24
Week 14 Nov 26
Topic: Thanksgiving break