EE 645
Machine Learning
Fall 2012
This course will give student a basic understanding and knowledge
of fundamental concepts of machine learning. Students will learn
about current research in area and conduct a project on topic of current
research.
Linear Algorithms for classification and regression: Perceptron
learning algorithm, Optimum margin classifiers, Logistic regression,
LMS learning algorithm, Least squares algorithms.
Kernel Methods: Optimal margin classifiers, Support Vector Machines,
least squares kernel methods, Radial Basis functions, Gaussian processes, on-line kernel learning algorithms.
More learning algorithms: Generative classifier (Naive Bayes),
Multilayer networks (error backpropagation algorithm),
ensemble learning (boosting, AdaBoost). mixture models, EM algorithm
Learning Theory:
Learning and generalization,
structural risk minimization, Dimensionality and generalization bounds.
Bayesian Learning: Bayesian decision theory, Bayesian networks,
graphical models, message passing and belief propagation.
Unsupervised Learning: Principal Component Analysis,
Independent Component Analysis, kernel PCA,
competitive learning algorithms, vector quantization.
Reinforcement Learning: Markov decision processes, dynamic
programming, Temporal Differences Learning, Q Learning, least
squares learning.
Selected References
T. Hastie, R. Tibshirani, and J. Friedman.
The Elements of Statistical Learning: Data Mining, Inference,
and Prediction, 2nd ed.. Springer, 2009.
Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.-T. Lin.
Learning from Data . AMLBook, 2012.
C. Bishop.
Pattern Recognition and Machine Learning.
Springer, 2006.
S. Haykin.
Neural Networks and Learning Machines 3rd Ed.
Prentice Hall, Englewood Cliffs, NJ, 2008.
R. Duda, P. Hart, and D. Stork.
Pattern Classification 2nd Ed.
Wiley, 2000.
J. Shawe-Taylor and N. Cristianini.
Kernel Methods for Pattern Analysis. Cambridge
University Press, 2004.
B. Scholkopf and A. Smola.
Learning with Kernels: Support Vector Machines, Regularization,
Optimization, and Beyond.
MIT Press, Cambridge, MA, 2002.
N. Cristianini and J. Shawe-Taylor.
An Introduction to Support Vector Machines: and Other
Kernel Based Learning Methods.
Cambridge University Press, Cambridge, UK, 2000.
D. Koller and N. Friedman.
Probabilistic Graphical Models Principles and Techniques.
MIT Press, 2009.
R. Sutton and A. Barto.
Reinforcement Learning: An Introduction (Adaptive computation and
machine learning).
Bradford Book, 1998.