When: Winter 2020, Tuesday & Thursday, 11:30 am - 12:55 pm

Where: Strathcona Anatomy & Dentistry M-1

→ Jump to schedule for lectures | tutorials | midterm

Instructor: Reihaneh Rabbany: rrabba@cs.mcgill.ca {add COMP551 in the title}

Office hours: Thursday, 1:30 pm - 2:30 pm @ McConnell Engineering Building (MC) 232

Course website: this page [www.reirab.com/comp55120.html] for information and MyCourses for engagement [e.g. quizzes]

→ There is a second section offered for this course (COMP 551-002) by Prof. Siamak Ravanbakhsh.

Teaching assistants: {joint with the second section}

FAQ

• Will there be recordings? No, but you can refer to the slides and assigned readings
• Will the two sections offer the same materials? That is the plan and assignments and mid-term will be jointly held, but the materials might or might not be covered in the same order, depth or pace.  
• Course/waiting list is full, can I register? The limit is based on room capacity [we are already above that], unfortunately, there is little I can do. This course is offered every semester. Hopefully, you can register for the next term.
• Do I meet the prerequisites? Please see below. If in doubt, see how well you follow the first few lectures.

Overview

This course covers a selected set of topics in machine learning and data mining, with an emphasis on good methods and practices for deployment of real systems. The majority of sections are related to commonly used supervised learning techniques, and to a lesser degree unsupervised methods. This includes fundamentals of algorithms on linear and logistic regression, decision trees, support vector machines, clustering, neural networks, as well as key techniques for feature selection and dimensionality reduction, error estimation and empirical validation.

Main Topics

Linear regression and linear classification

decision trees and ensemble methods

Support vector machines, kernels

Feature selection and dimensionality reduction

Performance evaluation, overfitting, cross-validation, bias-variance analysis, error estimation

Artificial neural networks and deep learning (e.g., RNNs and CNNs)

Unsupervised learning, clustering and Generative models

Please note that this syllabus is tentative and subject to change.

Relevant Textbooks

No required textbook but slides will cover chapters from the following books which can be used as reference materials.

[Bishop] Pattern Recognition and Machine Learning by Christopher Bishop (2007), available online 

[HTF] The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Trevor Hastie, Robert Tibshirani and Jerome Friedman (2009), available online

[Murphy] Machine Learning: A Probabilistic Perspective by Kevin Murphy (2012), available online through the library 

[GBC] Deep Learning  by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016), available online 

Prerequisites

This course requires programming skills (python) and basic knowledge of probabilities, calculus and linear algebra provided by courses similar to MATH-323 or ECSE-305. For more information see the course prerequisites and restrictions at McGill’s webpage.

Evaluation and Grading

The class grade will be based on the following components:

Weekly quizzes - 15% {online in mycourses} start on Jan 20th

Mini-projects - 50%  {group assignments}

Midterm examination - 35% {written on March 30th, 18:05-20:55, location: LEA 132 (section001) and ADAMs S AUD (section002)}

Date

Topic and Slides

Suggested Readings

Deadlines & deliverables

subject to change

1

Tue., Jan. 7

Syllabus & Intro to ML

Bishop Ch. 1 & 2, Murphy Ch. 1., HTF Ch. 1, ML review paper by Domingos (2012)

2

Thu., Jan. 9

ML Basics

Same as above

3

Tue., Jan. 14

Linear regression

Bishop Ch. 3, HTF Ch. 2-3

4

Thu., Jan. 16

Linear classification†

Bishop Ch. 4, HTF Ch. 4

5

Tue., Jan. 21

Tutorial in class

Math + Python

Mini-project 1 available

add/drop deadline (McGill calendar)

6

Thu., Jan. 23

Linear classification (cont.)

Bishop Ch. 4, HTF Ch. 4

7

Tue., Jan. 28

Naive Bayes†

Same as above

8

Thu., Jan. 30

Regularization†

Bishop Ch. 3

9

Tue., Feb. 4

Gradient Descent†

Murphy 8.5

10

Thu., Feb. 6

Evaluation†

HTF Ch. 7

11

Tue., Feb. 11

SVM†

Bishop Ch. 7, HTF Ch. 4 & 12

Mini-project 1 is due

12

Thu., Feb. 13

Decision Trees†

Bishop Ch. 14, HTF Ch. 9, visual demo

13

Tue., Feb. 18

Bootstrap & Bagging†

HTF Ch. 8, 10 & 15, Murphy Ch 16

14

Thu., Feb. 20

Boosting†

HTF 10,  visual demo 1,  visual demo 2

Mini-project 2 is available

15

Tue., Feb. 25

Boosting†(cont.)

HFT 10.10, Murphy 16.4

16

Thu., Feb. 27

Multilayer Perceptrons†

GBC 6,  visual demo

Tue., Mar. 3

Holiday break Mar. 2-6

Thu., Mar. 5

Holiday break Mar. 2-6

17

Tue., Mar. 10

MLP (cont.)† & Gradient Computation

GBC 6.5 and 8.2, nice video on backprop (from nn series), nice blog post

18

Thu., Mar. 12

Gradient Computation† (cont.)

Same as above, loss visualization

Mini-project 2 is due

19

Tue., Mar. 17

Class cancelled

20

Thu., Mar. 19

Class cancelled

21

Tue., Mar. 24

Class cancelled

Mini-project 3 is available

22

Thu., Mar. 26

Class cancelled

23

Tue., Mar. 31

Mini-project 3 is available

24

Thu., Apr. 2

25

Tue., Apr. 7

26

Thu., Apr. 9

Mini-project 3 is due (extended to April 14th)

Probability and Linear Algebra

Jan 21st, 11:35am-12:55pm, Strathcona Anatomy & Dentistry M-1

Jan 21st 6:05pm-7:55, RPHYS 112

Python/ NumPy

Jan 20th, 6:05pm-7:55pm, ENGMC 304

Jan 21st, 11:35am-12:55pm, Strathcona Anatomy & Dentistry M-1

https://www.python.org/

Scikit-learn

Feb 3rd, 6:05pm-7:25pm, ENGMC 304

Feb 4th, 4:05pm-5:25pm, BURN 1B24

Feb 6th, 6:05pm-7:55pm, ENGMC 304

https://scikit-learn.org/

PyTorch

Feb 24th, 6:05pm-7:55pm, MAASS 112

Feb 25th, 6:05pm-7:55pm, LEA 232

Feb 26th, 3:05pm-4:55pm, EDUC 624

https://pytorch.org/