# snrazavi/Machine_Learning_2018

Codes and Project for Machine Learning Course, Fall 2018, University of Tabriz

repo name | snrazavi/Machine_Learning_2018 |

repo link | https://github.com/snrazavi/Machine_Learning_2018 |

homepage | |

language | Jupyter Notebook |

size (curr.) | 34916 kB |

stars (curr.) | 93 |

created | 2018-10-16 |

license | |

# Machine Learning Course (Fall 2018)

Codes and Projects for Machine Learning Course, University of Tabriz.

# Contents:

### Chapter 1: Introduction (video)

- download slides in Persian (pdf)

## Supervised Learning

### Chapter 2: Regression

- Linear regression
- Gradient descent algorithm (video)
- Multi-variable linear regression
- Polynomial regression (video)
- Normal equation
- Locally weighted regression
- Probabilistic interpretation (video)
- Download slides in Persian (pdf)

### Chapter 3: Python and NumPy

- Python basics
- Creating vectors and matrices in
`numpy`

- Reading and writing data from/to files
- Matrix operations (video)
- Colon (:) operator
- Plotting using
`matplotlib`

(video) - Control structures in python
- Implementing linear regression cost function (video)

### Chapter 4: Logistic Regression (video)

- Classification and logistic regression
- Probabilistic interpretation
- Logistic regression cost function
- Logistic regression and gradient descent
- Multi-class logistic regression
- Advanced optimization methods
- Download slides in Persian (pdf)

#### Furthur Reading

- Artificial Intelligence: A Modern Approach (3rd Edition), pages 725-727
- An Introduction to Statistical Learning: with Applications in R, pages 130-137
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction, pages 119-128

### Chapter 5: Regularization (video)

- Overfitting and Regularization
- L2-Regularization (Ridge)
- L1-Regularization (Lasso)
- Regression with regularization
- Classification with regularization
- Download slides in Persian (pdf)

#### Furthur Reading

### Chapter 6: Neural Networks (video)

- Milti-class logistic regression
- Softmax classifier
- Training softmax classifier
- Geometric interpretation
- Non-linear classification
- Neural Networks (video: part 2)
- Training neural networks: Backpropagation
- Training neural networks: advanced optimization methods (video: part 3)
- Gradient checking
- Mini-batch gradient descent
- Download slides in Persian (pdf)

#### Demo:

#### Related Videos:

- Step by step Implementation of a multi-layer neural network in Python
- Backpropagation algorithm: Step by step example
- Activation functions (Sigmoid, Tanh, ReLU, PReLU, Maxout) and weight initialization methods
- How to solve problems using neural networks
- Dropout: A Simple Way to Prevent Neural Networks from Overfitting
- Neurons and how they work

#### Free Online Books:

- Neural Networks and Deep Learning; Michael Nielsen: This book is a very good place to start learning about neural networks and deep learning.
- Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville: For a more technical review of neural networks and deep learning, I recommend this book.

### Chapter 7: Support Vector Machines

- Motivation: optimal decision boundary (video: part 1)
- Support vectors and margin
- Objective function formulation: primal and dual
- Non-linear classification: soft margin (video: part 2)
- Non-linear classification: kernel trick
- Multi-class SVM
- Download slides in persian (pdf)

#### Demo:

#### Furthur Reading

## Unsupervided Learning

### Chapter 8: Clustering (video)

- Supervised vs unsupervised learning
- Clustering
- K-Means clustering algorithm (demo)
- Determining number of clusters: Elbow method
- Postprocessing methods: Merge and Split clusters
- Bisectioning clustering
- Hierarchical clustering
- Application 1: Clustering digits
- Application 2: Image Compression
- Download slides in Persian (pdf)

### Chapter 9: Dimensionality Reduction and PCA (video)

- Introduction to PCA
- PCA implementation in python
- PCA Applications
- Singular Value Decomposition (SVD)
- Downloas slides in Persian (pdf)

### Chapter 10: Anomally Detection (video: Part 1, Part 2)

- Intoduction to anomaly detection
- Some applications (security, manufacturing, fraud detection)
- Anoamly detection using probabilitic modelling
- Uni-variate normal distribution for anomaly detection
- Multi-variate normal distribution for anomaly detection
- Evaluation measures (TP, FP, TN, FN, Precision, Recall, F-score)
- Anomaly detection as one-class classification
- Classification vs anomaly detection
- Download slides in Persian (pdf)

### Chapter 11: Recommender Systems (video)

- Introduction to recommender systems
- Collaborative filtering approach
- User-based collaborative filtering
- Item-based collaborative filtering
- Similarity measures (Pearson, Cosine, Euclidian)
- Cold start problem
- Singular value decomposition
- Content-based recommendation
- Cost function and minimization
- Download slides in Persian (pdf)

## Other Useful Resources

- Optimization: Convex Optimization, Stephan Boyd, Stanford
- Linear algebra: pdf
- Calculus: Khan Accademy
- Probability: Khan Accademy

## Assignments:

- Regression and Gradient Descent
- Classification, Logistic Regression and Regularization
- Multi-Class Logistic Regression
- Neural Networks Training
- Neural Networks Implementing
- Clustering
- Dimensionallity Reduction and PCA
- Recommender Systems