EE 645
Machine Learning
Fall 2012

This course will give student a basic understanding and knowledge of fundamental concepts of machine learning. Students will learn about current research in area and conduct a project on topic of current research.

  • Linear Algorithms for classification and regression: Perceptron learning algorithm, Optimum margin classifiers, Logistic regression, LMS learning algorithm, Least squares algorithms.

  • Kernel Methods: Optimal margin classifiers, Support Vector Machines, least squares kernel methods, Radial Basis functions, Gaussian processes, on-line kernel learning algorithms.

  • More learning algorithms: Generative classifier (Naive Bayes), Multilayer networks (error backpropagation algorithm), ensemble learning (boosting, AdaBoost). mixture models, EM algorithm

  • Learning Theory: Learning and generalization, structural risk minimization, Dimensionality and generalization bounds.

  • Bayesian Learning: Bayesian decision theory, Bayesian networks, graphical models, message passing and belief propagation.

  • Unsupervised Learning: Principal Component Analysis, Independent Component Analysis, kernel PCA, competitive learning algorithms, vector quantization.

  • Reinforcement Learning: Markov decision processes, dynamic programming, Temporal Differences Learning, Q Learning, least squares learning.

    Selected References

  • T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed.. Springer, 2009.

  • Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.-T. Lin. Learning from Data . AMLBook, 2012.

  • C. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.

  • S. Haykin. Neural Networks and Learning Machines 3rd Ed. Prentice Hall, Englewood Cliffs, NJ, 2008.

  • R. Duda, P. Hart, and D. Stork. Pattern Classification 2nd Ed. Wiley, 2000.

  • J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.

  • B. Scholkopf and A. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, 2002.

  • N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines: and Other Kernel Based Learning Methods. Cambridge University Press, Cambridge, UK, 2000.

  • D. Koller and N. Friedman. Probabilistic Graphical Models Principles and Techniques. MIT Press, 2009.

  • R. Sutton and A. Barto. Reinforcement Learning: An Introduction (Adaptive computation and machine learning). Bradford Book, 1998.