Overview
This is a introductory machine learning (ML) course. The course in intended for developing foundations in machine learing and inference principles in applied areas such as natural language processing and data and text mining. The broader goal is to understand how machine learning tasks are carried out in the real world and how to build tools for solving them. Throughout the course, emphasis will be placed on understanding of concepts and tying specific techniques to specific real-world applications through hands-on experience.
Administrative details
Instructor office hours: M 2:30 - 3:30 PM
TA: Sadat Shahriar (sshahria@cougarnet.uh.edu)
TA office hours: T R 3-4 PM PGH 301
Prerequisites
The course requires some background in mathematics and sufficient programming skills. If you have taken and did well in one or more of the equivalent courses/topics such as Algorithms, Artificial Intelligence, Numerical methods, or have some background in probability/statistics, it will be helpful. The course however reviews and covers required mathematical and statistical foundations. Sufficient experience for building projects in a high level programming language (e.g., Java, Python) will prove beneficial.
Reading Materials
Textbooks:
SI: Statistical Inference, Casella and Berger. Cengage Learning; 2nd edition.
MLPP: Machien Learning - A Probabilistic Perspective. Kevin P. Murphy. MIT Press
WDM: Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data. Bing Liu; Springer, 1st Edition.
FSNLP: Foundations of Statistical Natural Language Processing, Chris Manning and Hinrich Schütze. MIT Press. Cambridge, MA: May 1999. Companion website for book.
Useful reference materials:
IML: Introduction to Machine Learning. Ethem Alpaydin. 2nd Edition. MIT Press.
Online resources (OR) per topic as appearing in the schedule below.
Lecture notes
Course materials (slides, lecture notes, etc.) (You may use 7zip to unpack)
NLP tools (POS Tagger, Chunker, Naive Bayes, etc.) and templates with linked libraries for research project
Assignment Due Dates and Grading
| Component | Contribution | Due date |
| Warm Up | 15% | TBA |
| Attendance | 5% | - |
| Exam 1 | 20% | TBA |
| Exam 2 | 30% | TBA |
| Project | 30% | TBA |
Schedule of topics
| Topic(s) | Resources: Readings, Slides, Lecture notes, Papers, Pointers to useful materials, etc. |
| Introduction
Course administrivia, plan, goals, ML Resources Key tasks in ML |
Required readings:
Lecture notes/slides Chapter 1 MLPP Chapter 1 IML |
| Math foundations I:Basics
Probability theory Conditional probability and independence |
Required Readings:
Lecture notes/slides Chapter 1 SI (Full reading recommended. Focus on topics covered in class and solved examples) Also refer to X.Zhu's notes on mathematical background |
| Math foundations II: Random varibales and Distributions
Random variables, density and mass fuctions Mean, Variance Common families of distributions Multiple random variables: joints and marginals |
Required Readings:
Lecture notes/slides Chapter 2 SI (Theorem 2.1.10, 2.2, 2.2.1, 2.2.2, 2.2.3, 2.2.5, 2.3.1, 2.3.2, 2.3.4, and select topics covered in class). Chapter 3 SI (All sections + worked out examples upto 3.4, and select topics covered in class). Chapter 4 SI (4.1, 4.1.1, 4.1.2, 4.1.3, 4.1.4, 4.1.5, 4.1.6, 4.1.10, 4.1.11, 4.1.12, 4.2.1, 4.2.2, 4.2.3, 4.2.4, 4.2.5). Optional Recommended reading/solved examples: OR03: Notes on Joint, marginals, worked out examples by S.Fan OR04: Tutorial on joints and marginals by M.Osborne [Contains NLP specific examples] |
| Supervised Learning I
Basic concepts: Data and features Decision tress Naive Bayes Classifier evaluation |
Required Readings: Chapter 3 WDM (3.1, 3.2, 3.3, 3.5, 3.6, 3.7.2)
F. Keller's tutorial on Naiye Bayes + notes of A.Moore for graph view (Slide 8) Programming resources, tools, libraries for projects and homeworks: Mallet, LingPipe |
| Supervised Learning II
Linear Regression Regularization Ridge Regression Support Vector Machines Feature selection |
Required Readings:
Chapter 7 MLPP (Seelct topics 7.1-7.3.1, 7.5, 7.5.4 covered in class) Chapter 8 MLPP (upto 8.3.1). Chapter 3 WDM (3.8, 3.10) Feature selection schemes: [Forman, 2003], [Mukherjee and Liu, 2010] Programming resources, tools, libraries for projects and homeworks: SVM: SVMLight, Boosted decision trees/Random forests: JForests, JBoost |
| Unsupervised Learning
Principal Component Analysis K-means clustering |
Required Readings: Chapter 4 WDM (4.1, 4.2, 4.4, 4.5, 4.6, 4.9).
Optional Recommended Reading: Clustering Analysis (Advanced) [Fred et al.], Modern Methods and Algorithmic Analyses [Müllner et al.] Programming resources, tools, libraries for projects and homeworks: Mallet, LingPipe, PU-Learning |
| Ensemble Learning
Bagging Boosting |
Required Readings: Chapter 3 WDM (3.10). |
| Markov models
Hidden markov model (HMM) A practical application: Part of speech tagging |
Required Readings:
Chapter 9 FSNLP (upto 9.4), Chapter 10 FSNLP (upto 10.2.2) OR10: (1) Lecture notes/slides by M. Marszalek on "A Tutorial on Hidden Markov Models by Lawrence R. Rabiner", (2) Toy problems by E.Lussier, (3) POS Tagging by Y.Choi Programming resources, tools, libraries for projects and homeworks: (1) HMMs and sequence taggers JAHMM: Implementation of an HMM in Java, Mallet, SVMHMM, CRF++ (2) POS Taggers OpenNLP, Stanford Parser, (Online version), Illinois Chunker, POS Tagger |
| Neural Networks
Artificial Neural Netorks Convolutional Neural Netorks Recurrent Neural Networks Long Short Term Memory Neural Networks Application: Word Embeddings |
Required Readings: TBA. |