CSCE 5218 – Deep Learning

Spring 2025    

Basic information:

Course description

This course aims at covering the basics of modern deep neural networks. In specific, the first part will introduce the fundamental concepts in neural networks including network architecture, activation function, loss, optimization, etc. Then, the second part will describe specific types of different deep neural networks such as convolutional neural networks (CNNs), recurrent neural networks (RNNs) and attention-based Transformer, as well as their applications in computer vision and natural language processing. In the final part we will briefly discuss some recent advanced topics in deep learning including graph neural networks, unsupervised representation learning, deep reinforcement learning, generative adversarial networks (GANs), etc. In this course, the hands-on practice of implementing deep learning algorithms (in Python) will be provided via homeworks and course project.

Textbooks

We will have required readings from the following textbook:

Besides, the following textbooks are useful as additional references: In addition to the textbooks, extra reading materials will be provided as we cover topics. Check out the course website regularly for updated reading materials.

Announcements

Links

Paper review list

Important: Read the requirements (click here) for paper review. Here is a review example for your reference. (Paper review lists will be gradually added.)

Paper review list 1 (due on 2/8):

  1. A Krizhevsky, I Sutskever, and G Hinton, ImageNet Classification with Deep Convolutional Neural Networks, NeurIPS, 2012.
  2. A Paszke, et al., PyTorch: An Imperative Style, High-Performance Deep Learning Library, NeurIPS, 2019.

Paper review list 2 (due on 2/20):

  1. K Simonyan and A Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, ICLR, 2015.
  2. K. He, X. Zhang, S. Ren, and J. Sun, Deep Residual Learning for Image Recognition, CVPR, 2016.

Paper review list 3 (due on 3/1):

  1. R. Girshick, J. Donahue, T. Darrell, and J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, CVPR, 2014.
  2. R. Girshick, Fast R-CNN, ICCV, 2015.

Paper review list 4 (due on 3/10):

  1. S. Ren, K. He, R. Girshick, and J. Sun, Faster R-CNN: Towards real-time object detection with region proposal networks, NIPS, 2015.
  2. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, You only look once: Unified, real-time object detection, CVPR, 2016.
  3. J. Long, E. Shelhamer, and T. Darrell, Fully Convolutional Networks for Semantic Segmentation, CVPR, 2015.

Paper review list 5 (due on 3/28):

  1. J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelling, NIPS Workshop, 2014.
  2. J. Donahue, L. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, Long-term recurrent convolutional networks for visual recognition and description, CVPR, 2015.

Paper review list 6 (due on 4/11):

  1. D. Bahdanau, K. Cho, and Y. Bengio, Neural machine translation by jointly learning to align and translate, ICLR, 2015.
  2. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, NIPS, 2017.
  3. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, ICLR, 2021.

Schedule and class notes (being updated)

Date Lecture Reading Note
Week 1
1/14 		Introduction - -
1/16 Machine learning overview Deep Learning Ch 1-5 -
Week 2
1/21 		Machine learning overview (cont) Deep Learning Ch 1-5 -
1/23 Neural network basics-1 Deep Learning Ch 4.2, 4.3, 6 -
Week 3
1/28 		Neural network basics-2 Deep Learning Ch 4.2, 4.3, 6 -
1/30 Deep neural network training-1 Deep Learning Ch 7, 8, 11 -
Week 4
2/4 		PyTorch Tutorial Deep Learning Ch 7, 8, 11 -
2/6 Deep neural network training-2 Deep Learning Ch 7, 8, 11 -
Week 5
2/11 		Deep neural network training-3 Deep Learning Ch 7, 8, 11 -
2/13 Convolutional Neural Networks (CNNs)
  • Convolution and Pooling
Deep Learning Ch 9 -
Week 6
2/18
  • Convolution and Pooling (cont.)
Deep Learning Ch 9 -
2/20
  • CNN Architectures and Applications
Deep Learning Ch 9 -
Week 7
2/25
  • CNN Architectures and Applications (cont.)
Deep Learning Ch 10 -
2/27 Recurrent Neural Networks (RNNs)
  • Basics and Architecture
Deep Learning Ch 10 -
Week 8
3/4
3/6 		Project Proposal Preparation - -
3/11-3/13 Spring Break (no classes) - Project proposal due to 3/13
3/18
  • Basics and Architecture (cont.)
Deep Learning Ch 10 -
3/20
  • Applications
Deep Learning Ch 10 -
3/25 Transformers
  • Background
  • Self- and Cross-attention
Ref 1, Ref 2, Ref 3, Ref 4 (a blog), Ref 5 -
3/27
  • Self- and Cross-attention (cont.)
  • Applications beyong language
Ref 1, Ref 2, Ref 3, Ref 4 (a blog), Ref 5 -
4/1 Advanced Topics
  • Graph Convolutional Networks
  • Self-supervised Learning
Ref 1, Ref 2 -
4/3
  • Reinforcement Learning
Ref 1, Ref 2, Ref 3 -
4/8
  • Generation
  • Generative Adversarial Networks
Ref 1 -
4/10
  • Bias and Ethics (optional)
- -
4/15-4/29 Project Presentation - Project report due to 4/21
Final Exam Date: 10:00 am-12:00 pm on May 8, at K110 - -