imported>Bbauwens: Migrated current public revision from wiki.cs.hse.ru

2022-12-16T11:35:59Z

Migrated current public revision from wiki.cs.hse.ru

Новая страница

== General Information ==

[https://www.dropbox.com/s/a8dd33ousa76qad/grading.pdf?dl=0 Grading]

Teachers: [https://www.hse.ru/en/org/persons/160550073 Bruno Bauwens] and [https://www.hse.ru/en/org/persons/225553845 Nikita Lukianenko]

Lectures: Saturday 14:40 - 16:00. The lectures are in zoom.

Seminars: Tuesday 16:20 - 17:40. The seminars are [https://meet.google.com/ber-yzns-hxz here] in google.meet.

Practical information on a telegram group.

The course is similar [http://wiki.cs.hse.ru/Statistical_learning_theory_2020 last year], except for the order of topics and part 3.

== Problems exam ==

Dec 22, 12:00 -- 15:30

During the exam 
-- You may consult notes, books and search on the internet 
-- You may not interact with other humans (e.g. by phone, forums, etc)

== Colloquium ==

Saturday December 11

[https://www.dropbox.com/s/u8hyo1omvaoujle/colloqQuest.pdf?dl=0 rules and list of questions] (version Dec 10)

== Homeworks ==

Email to brbauwens-at-gmail.com. Start the subject line with SLT-HW. [https://www.dropbox.com/s/prsmzhtr5p5uome/scores.pdf?dl=0 Results]

Deadline before the lecture, every other lecture.

25 Sept: see problem lists 1 and 2 
09 Oct: see problem lists 3 and 4 
29 Oct: see problem lists 5 and 6 
13 Nov: see problem lists 7 and 8 
30 Nov, 08:00 [extended]: see problem lists 9 and 10

== Course materials ==

{| class="wikitable"
|-
! Video !! Summary !! Slides !! Lecture notes !! Problem list !! Solutions
|-
|
|| ''Part 1. Online learning''
|-
| [https://drive.google.com/file/d/1WL9LSNDD1B_q6LdpfDQ8BPluNfhjWrD9/view?usp=sharing 4 Sept]
|| Lecture: philosophy. Seminar: the online mistake bound model, the weighted majority, and perceptron algorithms [https://drive.google.com/drive/folders/1NXiLbhmO2Ml7jFmnLtjqhOgCoHg7yn9T?usp=sharing movies]
|| [https://www.dropbox.com/s/uk9awkfa827pmtf/01allSlides.pdf?dl=0 sl01]
|| [https://www.dropbox.com/s/uvsfzb997kantoa/00book_intro.pdf?dl=0 ch00] [https://www.dropbox.com/s/6ah70h5loyrz5lx/01book_onlineMistakeBound.pdf?dl=0 ch01]
|| [https://www.dropbox.com/s/aoma8ma8mkd3885/01sem.pdf?dl=0 01prob (9 Sept)]
|| [https://www.dropbox.com/s/sqzqlrtzr2nu8cq/01sol.pdf?dl=0 01sol]
|-
| [https://drive.google.com/file/d/16OoCqhh16BKQzyF-HM8RozigyJ3BBVxA/view?usp=sharing 11 Sept]
|| The perceptron algorithm in the agnostic setting. Kernels. The standard optimal algorithm.
|| [https://www.dropbox.com/s/sy959ee81mov5cr/02slides.pdf?dl=0 sl02]
|| [https://www.dropbox.com/s/0029k15cbnxj2v1/02book_sequentialOptimalAlgorithm.pdf?dl=0 ch02] [https://www.dropbox.com/s/eggk7kctgox8aza/03book_perceptron.pdf?dl=0 ch03]
|| [https://www.dropbox.com/s/415nws7qi589bme/02sem.pdf?dl=0 02prob (23 Sept)]
|| [https://www.dropbox.com/s/ofcctflbnxt0kx3/02sol.pdf?dl=0 02sol]
|-
| 18 Sept (rec to do)
|| Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory.
|| [https://www.dropbox.com/s/a60p9b76cxusgqy/03slides.pdf?dl=0 sl03]
|| [https://www.dropbox.com/s/ytl6q83q6gkax3w/04book_predictionWithExperts.pdf?dl=0 ch04] [https://www.dropbox.com/s/l11afq1d0qn6za7/05book_introProbability.pdf?dl=0 ch05]
|| [https://www.dropbox.com/s/nsrcy3yxgey67lp/03sem.pdf?dl=0 03prob(30 Sept)]
|| [https://www.dropbox.com/s/bg9nd01h1fhzjsi/03sol.pdf?dl=0 03sol]
|-
|
|| ''Part 2. Risk bounds for binary classification''
|-
| [https://drive.google.com/file/d/1RHz8NgfianUQFlx8VswjiiPRvt0DoBvc/view?usp=sharing 25 Sept]
|| Sample complexity in the realizable setting, simple examples and bounds using VC-dimension
|| [https://www.dropbox.com/s/pi0f3wab1xna6d7/04slides.pdf?dl=0 sl04]
|| [https://www.dropbox.com/s/8xrgcugs4xv2r2p/06book_sampleComplexity.pdf?dl=0 ch06]
|| [https://www.dropbox.com/s/7qn2yz5fxc93rez/04sem.pdf?dl=0 04prob]
|| [https://www.dropbox.com/s/xm3nhgj5d6h49nz/04sol.pdf?dl=0 04sol]
|-
| [https://drive.google.com/drive/folders/1jjyJ3eIaed64ogpR11g8M44IOikt5Mj2?usp=sharing 2 Oct]
|| Growth functions, VC-dimension and the characterization of sample comlexity with VC-dimensions
|| [https://www.dropbox.com/s/rpnh6288rdb3j8m/05slides.pdf?dl=0 sl05]
|| [https://www.dropbox.com/s/ctc48w1d2vvyiyt/07book_growthFunctions.pdf?dl=0 ch07] [https://www.dropbox.com/s/jofixf9tstz0f8z/08book_VCdimension.pdf?dl=0 ch08]
|| [https://www.dropbox.com/s/zbyqxy3qp3pz79i/05sem.pdf?dl=0 05prob]
|| [https://www.dropbox.com/s/a8efm18dof2zeox/05sol.pdf?dl=0 05sol]
|-
| [https://drive.google.com/file/d/17zynIg_CZ6cCNBig5QXmBx7VFS8peyuU/view?usp=sharing 9 Oct]
|| Risk decomposition and the fundamental theorem of statistical learning theory
|| [https://www.dropbox.com/s/jxijka88vfanv5n/06slides.pdf?dl=0 sl06]
|| [https://www.dropbox.com/s/r44bwxz34qj98gg/09book_riskBounds.pdf?dl=0 ch09]
|| [https://www.dropbox.com/s/x87txc8v5v6u8vb/06sem.pdf?dl=0 06prob]
|| [https://www.dropbox.com/s/ydlqu8oce3xj6ix/06sol.pdf?dl=0 06sol]
|-
| 16 Oct
|| Bounded differences inequality and Rademacher complexity
|| [https://www.dropbox.com/s/kfithyq0dgcq6h8/07slides.pdf?dl=0 sl07]
|| [https://www.dropbox.com/s/5quc1jfkrvm3t71/10book_measureConcentration.pdf?dl=0 ch10] [https://www.dropbox.com/s/km0fns8n3aihauv/11book_RademacherComplexity.pdf?dl=0 ch11]
|| [https://www.dropbox.com/s/d1rsxceqmbk5llw/07sem.pdf?dl=0 07prob]
|| [https://www.dropbox.com/s/sftaa8b92ru3ii5/07sol.pdf?dl=0 07sol]
|-
| [https://drive.google.com/file/d/1L-BeDxhoHcoDrdlVTlfoMFwnWXKV46cr/view?usp=sharing 30 Oct]
|| Simple regression, support vector machines, margin risk bounds, and neural nets
|| [https://www.dropbox.com/s/0xrhe4732d0jshb/08slides.pdf?dl=0 sl08]
|| [https://www.dropbox.com/s/cvqlwst3e69709t/12book_regression.pdf?dl=0 ch12] [https://www.dropbox.com/s/dwwxgriiaj4efn0/13book_SVM.pdf?dl=0 ch13]
|| [https://www.dropbox.com/s/qqdbrh2ll0dv03a/08sem.pdf?dl=0 08prob]
|| [https://www.dropbox.com/s/9o8fyd0ff735hxu/08sol.pdf?dl=0 08sol]
|-
| [https://youtu.be/9FhFxLHR4eE 6 Nov]
|| Kernels: risk bounds, RKHS, representer theorem, design
|| [https://www.dropbox.com/s/nhqtbekclekf6k7/09slides.pdf?dl=0 sl09]
|| [https://www.dropbox.com/s/bpb9ijn2p7k19j3/14book_kernels.pdf?dl=0 ch14]
|| [https://www.dropbox.com/s/d2dmh017lw207ns/09sem.pdf?dl=0 09prob] (Nov 23)
|| [https://www.dropbox.com/s/2wq9mxrqchsqujr/09sol.pdf?dl=0 09sol]
|-
| [https://youtu.be/ZBHe5RhTuzI 13 Nov]
|| AdaBoost and risk bounds
|| [https://www.dropbox.com/s/umum3kd9439dt42/10slides.pdf?dl=0 sl10]
|| Mohri et al, chapt 7
|| [https://www.dropbox.com/s/j8s197e0mjv9qla/10sem.pdf?dl=0 10prob] (Nov 23)
|| [https://www.dropbox.com/s/7lw1u8750k7s8qt/10sol.pdf?dl=0 10sol]
|-
|
|| ''Part 3. Other topics''
|-
| [https://youtu.be/L4o7dXcaQrk 20 Nov]
|| Clustering
|| [https://www.dropbox.com/s/5a9flvg95iihz7m/11slides.pdf?dl=0 sl11]
|| Mohri et al, ch7; [https://people.csail.mit.edu/dsontag/courses/ml12/slides/lecture14.pdf lecture]
|| 
|| 
|-
| [https://youtu.be/FN6l4Ceq5lE 27 Nov]
|| Dimensionality reduction and the Johnson-Lindenstrauss lemma
|| [https://www.dropbox.com/s/wbgwwk7a9mjo1bv/12slides.pdf?dl=0 sl12]
|| Mohri et al, ch15; [https://ramanlab.wustl.edu/Lectures/Lecture12_LDA_CCA.pdf lecture]
|| [https://www.dropbox.com/s/c5anx2htaw9rslr/12sem.pdf?dl=0 12prob]
||
|-
| 4 Dec
|| No lecture
||
||
||
||
|-
| 11 Dec
|| Colloquium
||
||
||
||
|}



The lectures in October and November are based on the book:
Foundations of machine learning 2nd ed, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2018. This book can be downloaded from http://gen.lib.rus.ec/ .

== Office hours ==

{| class="wikitable"
|-
! Person !! Monday !! Tuesday !! Wednesday !! Thursday !! Friday !!
|-
| [https://www.hse.ru/en/org/persons/160550073 Bruno Bauwens], [https://zoom.us/j/5579743402 Zoom] || || 12h30-14h30 || || || 14h-20h || Room S834 Pokrovkaya 11
|-
| [https://www.hse.ru/org/persons/225553845 Nikita Lukianenko], [https://t.me/vaulty Telegram] || || 14h30-16h30 || 14h30-16h30 || || || Room S831 Pokrovkaya 11
|-
|}

It is always good to send an email in advance. Questions and feedback are welcome.

Statistical learning theory 2021 - История изменений

imported>Bbauwens: Migrated current public revision from wiki.cs.hse.ru