Kullback-Leibler Divergence

less than 1 minute read


Topics: Kullback-Leibler Divergence

One of the points on my long ‘stuff-you-have-to-look-at’ list is the Kullback-Leibler divergence. I finally took the time to take a detailed look at this topic. Due to problems with displaying the math formulas I decided to create a jupyter notebook with the blog post. You can find it here.

There is also a PDF version of the explanations which has by far the nicest looks (e.g. all the equations are aligned). You can download it here.

Taking a detailed look at the KL divergence was definitely worth it! Next, I want to learn more about variational inference.