**CAP5610
Machine Learning**

Term: Spring 2017

Time: Th 6-8:50pm

Room: Rm726, Office of Continuing Education, Innovative Center (3280 Progress Drive, Suite 700, Orlando, FL 32826)

Instructor: Guo-Jun Qi, guojun.qi@ucf.edu

**Pre-requisites**

Calculus, linear algebra and probability theory

**Background
**

In this course, we will begin with the basic concepts in machine learning, in the context of several classic topics from supervised learning (classification), unsupervised learning (model fitting, clustering) to feature learning, dimensionality reduction etc. We will also discuss advanced topics on learning theory, graphical models, and dynamic Bayesian model if time allows.

**Books**:

Required textbook: Pattern Recognition and Machine Learning (PRML), C. Bishop, Springer, 2006.

Other suggested books:

Machine Learning, T. Mitchell, McGraw-Hill, 1997.

The Elements of Statistical Learning: Data
Mining, Inference, and Prediction, 2^{nd} Edition, T. Hastie, R. Tibshirani, J. Friedman, Springer, 2011.

Probabilistic Graphical Models: Principles and Techniques, D. Koller, and N. Friedman, The MIT Press, 2009.

**Covered
topics **(subject to change)**:**

1. Introduction to machine learning: feature extraction, examples of machine learning problems, training and test protocol, datasets, KNN

2. Naive Bayesian classifier

3. Linear regression and classification

4. Support vector machine and kernel method

5. Neural Networks and back propagation

6. Unsupervised learning problems: clustering, PCA, LDA, CCA etc.

7. Modeling fitting and EM algorithm

8. Graphical model: Baysian networks, Markov random fields, approximation inference, variational method, sampling, loopy belief propagation

9. Advanced topics: matrix factorization, metric learning, latent models, online learning, active learning, latent models, sparse coding, nonparametric Bayesian model etc. (will cover as many as possible if time allows)

**Grading
policy:**

Homework and machine problems will account for 30% of the grade. There will be two exams (50%) and a final project (20%). There is no final exam.

**Final
Project:**

Students can perform individual final project, or form groups of up to three people to collaborate on the final project. A final project can be a survey (paper review) on a certain machine learning topic, or a proposed project to solve a machine learning problem. Instructor will also suggest some topics for the project. A project proposal shall be submitted to instructor for review before it is approved. The proposal must specify who will be joining in the project, the roles of each member, and description of the proposed survey topic or problem. Project report will be graded. Project presentation will be arranged if time allows.

**Machine
Problems:**

The course is not intended to teach a programming language, so students can use any languages to solve the machine problems (e.g., C, C++, Java, Matlab, Python, Octave). You are required to submit a report, which shall include a brief description of how you implement the algorithm in the language you choose, the parameter setting, your test protocol, as well as the result you obtain. Source codes shall be submitted along with the report. Submit reports and source codes to cap5610ucf@gmail.com.

**Lecture
Nodes:**

·
January 12, Lecture 01:
Introduction to Machine Learning [ **pdf** ]

Reading assignment: Sec 1.2, Sec 1.4 of PRML.

Link to MNIST dataset in
different formats: [ **MNIST a** ] [ **MNIST
b** ] (two datasets are the same)

·
January 19, Lecture 02: Review
of Probability Theory [ **pdf** ]

Reading assignment Chap 2 and Appendix B of PRML.

·
January 26, Lecture 03: Bayes
Classifier [ **pdf** ]
Bayes Classifier II [ **pdf** ]

Reading assignment Section 1.5 of PRML.

·
February 2, Lecture 04:
Logistic Regression [ **pdf** ]

Chapter 4 of PRML

·
February 9, Lecture 05: Linear
Regression [ **pdf** ]

Chapter 4 of PRML

·
February 16, Lecture 06:
Support Vector Machines [ **pdf** ]

·
February 23, Lecture 07:
Support Vector Machines II [ **pdf** ]

·
March 2, Lecture 08: Neural
Networks [ **pdf** ]

·
March 9, Lecture 09: Deep
Learning [ **pdf** ]

·
April 6, Lecture 10:
Dimensionality Reduction [ **pdf**
]

·
April 13, Lecture 11:
Clustering analysis: K-means [ **pdf** ]
and GMM [ **pdf** ]

·
April 20, Lecture 12: Boosting
[ **pdf** ]