Continuing from the previous blog post, we will carry on with the core message of the two part series: optimal transport gives us calculus on the space of probability distributions. Just in case you were not already convinced by the first post — perhaps because Langevin diffusion is not everyone’s cup of tea — then neural networks might be the keyword that piques your interest.
In my opinion, optimal transport (OT) is a seriously underrated topic. I think part of the reason is the way OT is often introduced: as an optimization problem or a metric on probability distributions. While these are interesting to study for their own sake, OT presents a tool much more powerful and arguably even fundamental to the study of probability. In a series of two posts, I’m going to present a message that is well understood by experts but often missed by the uninitiated:
Dyson Brownian motion [Dy62] is best known to characterize the eigenvalues of special random matrices [Ta12]. Most interestingly, it is also equal in distribution to \(n\) independent Brownian motions conditioned to not intersect [Gr99]. In a topics course by Bálint Virág, I came across a proof of this result that is just too clean for this type of calculations. After picking up my jaw from the ground months later, I finally decided to write up this surprisingly elegant proof.
Nobody has time to read an 80 page paper [LE20]. Therefore I doubt most readers realized the manifold Langevin algorithm paper actually contains a novel technique for establishing functional inequalities. And I really doubt anyone had time to interpret the intuitive consequences of such results on perturbed gradient descent, and definitely not extending the Kannan-Lovász-Simonovits (KLS) conjecture [LV18] - which brings me to write this blog post.
Equivalent representation results contribute not only a connection between different concepts, but also a new set of proof techniques. Indeed, stochastic analysis has offered a number of alternative proofs to many problems. Occasionally the proof can simplify drastically. In this post, we will discuss a particularly elegant application by Auffinger and Chen (2015), for an otherwise very difficult problem in spin glass.
In a similar sense to line integrals, stochastic calculus extends the classical tools to working with stochastic processes. One of the most elegant and useful result is the change of variable formula for stochastic integrals, commonly known as Itô’s Lemma (see end of this post for a discussion on Doeblin’s contribution). While this lemma is quite easy to use, the proof usually relies heavily on technical lemmas, hence difficult to develop intuition, especially for the first time reader.
While studying two seemingly irrelevant subjects, probability theory and partial differential equations (PDEs), I ran into a somewhat surprising overlap: the Poincaré inequality. On one hand, it is not out of the ordinary for analysis based subjects to share inequalities such as Cauchy-Schwarz and Hölder; on the other hand, the two forms of Poincaré inequality have quite different applications.