Mathematics for machine learning
Information theory basics
Entropy, cross-entropy, KL divergence, and where they appear in losses.
5 lessons
Follow in order
Use the button below to sign in and unlock lessons.
Your path
Lessons in sequence
Work through these in order—each lesson builds on the previous one.
-
Lesson 1 of 5
Entropy and uncertainty
-
Lesson 2 of 5
Cross-entropy
-
Lesson 3 of 5
KL divergence
-
Lesson 4 of 5
Mutual information
-
Lesson 5 of 5
Information theory in loss functions