ML202Semester 43 (2-0-2)Major

Introduction to Machine Learning

Syllabus

Unit 1: Foundations of Machine Learning

Core concepts and taxonomy of machine learning (supervised, unsupervised, reinforcement learning), Bias-variance tradeoff and model capacity, Overfitting, underfitting, and the No Free Lunch theorem, Evaluation metrics (accuracy, precision, recall, F1-score, ROC-AUC, confusion matrix), Cross-validation strategies (k-fold, stratified, time-series), Feature engineering fundamentals (scaling, encoding, feature selection).

Unit 2: Linear Models and Regression

Linear regression assumptions and ordinary least squares (OLS) solution, Gradient descent variants (batch, stochastic, mini-batch), Regularization techniques (L1/Lasso, L2/Ridge, Elastic Net), Logistic regression for binary classification, Probability interpretation and maximum likelihood estimation, Multinomial logistic regression and softmax, Model diagnostics (residual analysis, learning curves).

Unit 3: Nonlinear Models and Decision Boundaries

Decision trees (CART algorithm, entropy, Gini impurity), Tree ensemble methods (bagging, random forests), Gradient boosting machines (XGBoost fundamentals), Support Vector Machines (maximal margin separator, kernel trick), Kernel functions (linear, polynomial, RBF), Nonlinear separability and the curse of dimensionality.

Unit 4: Unsupervised Learning and Clustering

K-means clustering algorithm and limitations, Hierarchical clustering (agglomerative, divisive), Density-based clustering (DBSCAN), Dimensionality reduction techniques (PCA, t-SNE, UMAP), Anomaly detection methods (isolation forest, one-class SVM), Association rule mining (Apriori, FP-growth).

Unit 5: Model Selection, Optimization, and Deployment

Hyperparameter optimization (grid search, random search, Bayesian optimization), Ensemble learning principles (stacking, voting), Model interpretability techniques (SHAP, LIME, partial dependence plots), Introduction to ML pipelines and cross-validation pitfalls, MLOps concepts (model versioning, monitoring, retraining), Practical considerations for production ML systems.

Skills

Data StructuresAlgorithmsMachine LearningStatisticsRoboticsGISSemiconductor Design

Structure

Semester: 4

Credits: 3 (2-0-2)

Category: Major

Practicals

Lab 1: ML Workflow and Evaluation Metrics: Focuses on implementing complete ML pipeline with data splitting, model training, and comprehensive evaluation using multiple metrics and cross-validation techniques.
Lab 2: Linear Regression Implementation: Focuses on implementing linear regression from scratch using gradient descent, comparing with scikit-learn implementation, and analyzing convergence behavior.
Lab 3: Logistic Regression and Classification: Focuses on binary classification problems, implementing logistic regression with regularization options, and interpreting probability outputs with ROC curves.
Lab 4: Decision Trees and Feature Importance: Focuses on building decision trees, visualizing tree structures, extracting feature importance, and pruning strategies to control overfitting.
Lab 5: Random Forests Ensemble: Focuses on implementing bagging with decision trees and comparing performance with single decision trees across multiple datasets.
Lab 6: SVM with Different Kernels: Focuses on experimenting with linear vs. nonlinear SVMs using various kernel functions and understanding the kernel trick through visualization.
Lab 7: K-means Clustering Analysis: Focuses on clustering datasets with known structures, evaluating cluster quality using silhouette scores, and handling initialization sensitivity.
Lab 8: Dimensionality Reduction Visualization: Focuses on applying PCA, t-SNE, and UMAP to high-dimensional datasets and creating insightful 2D/3D visualizations of data structure.
Lab 9: Hyperparameter Tuning Pipeline: Focuses on implementing grid search and random search for multiple algorithms, creating learning curves, and selecting optimal model configurations.
Lab 10: End-to-End ML Project: Focuses on complete ML project from data ingestion through model deployment, including feature engineering, model selection, evaluation, and basic MLOps practices.