| Course code | CSCI4250 |
| Course title | Online Algorithms for Machine Learning and Optimization 機器學習和優化的在線算法 |
| Course description | This course aims to cover topics in online learning and online optimization. Typical topics include multi-armed bandit (MAB) problems in various settings, online convex optimization (OCO) problems such as online linear regression, online classification, and certain general reinforcement learning problems. Different algorithms will be introduced to solve these problems and analysis of the performance and efficiency will be provided. 本科旨在涵蓋在線學習和在線優化中的一些問題。典型的題目包括各類模型下的MAB問題，在線線性回歸等在線凸優化問題，在線分類問題，以及某些一般的增強學習問題。課程會介紹對這些問題的不同的算法，並提供其性能效率的分析。 |
| Unit(s) | 3 |
| Course level | Undergraduate |
| Semester | 1 or 2 |
| Grading basis | Graded |
| Grade Descriptors | A/A-: EXCELLENT – exceptionally good performance and far exceeding expectation in all or most of the course learning outcomes; demonstration of superior understanding of the subject matter, the ability to analyze problems and apply extensive knowledge, and skillful use of concepts and materials to derive proper solutions. B+/B/B-: GOOD – good performance in all course learning outcomes and exceeding expectation in some of them; demonstration of good understanding of the subject matter and the ability to use proper concepts and materials to solve most of the problems encountered. C+/C/C-: FAIR – adequate performance and meeting expectation in all course learning outcomes; demonstration of adequate understanding of the subject matter and the ability to solve simple problems. D+/D: MARGINAL – performance barely meets the expectation in the essential course learning outcomes; demonstration of partial understanding of the subject matter and the ability to solve simple problems. F: FAILURE – performance does not meet the expectation in the essential course learning outcomes; demonstration of serious deficiencies and the need to retake the course. |
| Learning outcomes | At the end of the course of studies, students will have acquired the ability to 1. understand typical problems of online learning and online optimization and to model certain practical problems into them; 2. design effective algorithms to solve the problems in online learning and online optimization; 3. analyse performance and efficiency of the algorithms. |
| Assessment (for reference only) |
Essay test or exam ：50% Homework or assignment ：20% Project ：30% |
| Recommended Reading List | 1. Online Learning and Online Convex Optimization, Shai Shalev-Shwartz, Foundations and Trends in Machine Learning, Vol. 4, No. 2, pages 107– 194, 2011. 2. Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Sebastien Bubeck and Nicolo Cesa-Bianchi, Foundations and Trends in Machine Learning, Vol. 5, No. 1, pages 1–122, 2012. 3. A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning, Alborz Geramifard, Thomas J. Walsh, Stefanie Tellex, Girish Chowdhary, Nicholas Roy, Jonathan P. How, Foundations and Trends in Machine Learning, Vol. 6, No. 4, pages 375–451, 2013. 4. Introduction to Online Convex Optimization, Elad Hazan, Foundations and Trends in Optimization, Vol. 2, No. 3-4, pages 157–325, 2015 |
| CSCIN programme learning outcomes | Course mapping |
| Upon completion of their studies, students will be able to: | |
| 1. identify, formulate, and solve computer science problems (K/S); | |
| 2. design, implement, test, and evaluate a computer system, component, or algorithm to meet desired needs (K/S); |
T |
| 3. receive the broad education necessary to understand the impact of computer science solutions in a global and societal context (K/V); | |
| 4. communicate effectively (S/V); |
|
| 5. succeed in research or industry related to computer science (K/S/V); |
|
| 6. have solid knowledge in computer science and engineering, including programming and languages, algorithms, theory, databases, etc. (K/S); | T |
| 7. integrate well into and contribute to the local society and the global community related to computer science (K/S/V); | T |
| 8. practise high standard of professional ethics (V); | |
| 9. draw on and integrate knowledge from many related areas (K/S/V); |
|
| Remarks: K = Knowledge outcomes; S = Skills outcomes; V = Values and attitude outcomes; T = Teach; P = Practice; M = Measured | |