ML301Semester 53 (2-0-2)Major

Deep Learning & Neural Networks

Syllabus

Unit 1: Neural Networks Fundamentals

Perceptron and multi-layer perceptron (MLP), Activation functions (sigmoid, tanh, ReLU, Leaky ReLU, Swish), Forward propagation and backpropagation algorithm derivation, Gradient descent optimization (SGD, momentum, AdaGrad, RMSprop, Adam), Vanishing/exploding gradients problem, Weight initialization strategies (Xavier, He), Universal approximation theorem.

Unit 2: Convolutional Neural Networks

Motivation for CNNs in image processing, Convolution operation and feature maps, Pooling layers (max, average, global), CNN architectures (LeNet, AlexNet, VGG, ResNet residual connections), Transfer learning and fine-tuning strategies, Data augmentation techniques, Batch normalization and layer normalization, Object detection fundamentals (sliding window, region proposals).

Unit 3: Recurrent Networks and Sequence Models

RNN architecture and vanishing gradient problem, Long Short-Term Memory (LSTM) cells and gates, Gated Recurrent Units (GRU), Bidirectional RNNs, Sequence-to-sequence models with attention mechanisms, Beam search decoding, Transformer architecture (self-attention, multi-head attention, positional encoding), BERT and GPT model families.

Unit 4: Advanced Architectures and Generative Models

Generative Adversarial Networks (GANs) - minimax game, training stability, DCGAN improvements, Variational Autoencoders (VAEs) - encoder-decoder, KL divergence, Diffusion models fundamentals (forward/reverse diffusion process, DDPM), Autoencoders and dimensionality reduction, Graph Neural Networks (GNN) introduction, Model compression techniques (pruning, quantization).

Unit 5: Training, Optimization, and Deployment

Loss functions (cross-entropy, MSE, Dice, contrastive losses), Learning rate schedules and schedulers, Gradient clipping, Mixed precision training, Distributed training strategies (data parallel, model parallel), Overfitting prevention (dropout, early stopping, data augmentation), Model evaluation for deep learning (confusion matrix, precision-recall curves), TensorRT/ONNX deployment, Edge deployment considerations.

Skills

Data StructuresAlgorithmsComputer NetworksDeep LearningCloud ComputingDatabasesRobotics

Structure

Semester: 5

Credits: 3 (2-0-2)

Category: Major

Practicals

Lab 1: MLP Implementation and Training: Focuses on implementing MLP from scratch, experimenting with activation functions and optimizers, visualizing decision boundaries.
Lab 2: CNN for Image Classification: Focuses on building CNNs for CIFAR10/MNIST, implementing convolution/pooling layers, transfer learning from pretrained models.
Lab 3: RNN/LSTM for Sequence Prediction: Focuses on character-level text generation and time-series forecasting using LSTM/GRU networks.
Lab 4: Attention and Transformer Basics: Focuses on implementing self-attention mechanisms and simple transformer encoder for text classification.
Lab 5: GAN Implementation: Focuses on training DCGAN for MNIST digit generation, analyzing training dynamics and mode collapse.
Lab 6: VAE and Diffusion Model Sampling: Focuses on implementing VAE for image reconstruction and basic diffusion model denoising process.
Lab 7: Object Detection Pipeline: Focuses on using YOLO/SSD for object detection, evaluating mAP metrics on custom datasets.
Lab 8: Advanced Training Techniques: Focuses on implementing learning rate scheduling, gradient clipping, mixed precision training.
Lab 9: Model Optimization and Compression: Focuses on pruning, quantization, and knowledge distillation for deployment-ready models.
Lab 10: End-to-End DL Project: Focuses on complete pipeline from data collection through model training, evaluation, and deployment using cloud services.