“Mathematics knows no races or geographical boundaries; for mathematics, the cultural world is one country.”
The subtleties of computing quantiles - June 21, 2020
Computing quantiles is a good way to summarize the distribution of a numerical dataset. But confusingly there are nearly a dozen different definitions of quantile that can all claim to be correct, and in all cases it is difficult to actually compute quantiles for very large datasets. I will explain why there are so many definitions, and compare a couple of different strategies for doing computations at scale.
Working with Python Project Directories - March 05, 2020
This short post contains some tips and tricks for dealing with complicated Python project directories in an organized way.
Jensen's Inequality and Statistics - September 01, 2019
Jensen's inequality is a beautiful little workhorse lemma from convex geometry. I have stumbled into it an number of times recently as I have been reading and writing about statistics and information theory, so I decided to split it off into its own blog post. After introducing the inequality (and proving that it holds), I will give a few snappy applications to basic statistics, including the classical inequality between arithmetic, geometric, and harmonic means as well as a basic inequality relating the mean, median, and standard deviation.
The Radon-Nikodym Theorem - May 27, 2019
The Radon-Nikodym theorem is a workhorse result in measure theory, with numerous applications to probability dynamics (such as the existence of conditional expectations and the existence of KL-divergence). I will give a simple proof using Hilbert spaces.
Gibbs' Inequality - May 27, 2019
It is a fairly standard fact that relative entropy (KL-divergence) is positive definite, but I was unsatisfied with the proofs of this fact that I saw when I glanced through the literature. In this post I will provide a complete proof which works on a general probability space.