Machine Learning through the lens of sub-Riemannian Geometry

In this talk we illustrate the link between Deep Neural Networks and flows induced by control systems (Neural ODEs), and we relate the "expressivity'' of a Residual Neural Network (ResNets) to the controllability properties of the corresponding Neural ODE in the space of diffeomorphisms. In case of control-linear Neural ODEs, a sub-Riemannian structure emerges. We show how the Lie Algebra Strong Approximating Property (see [Agrachev & Sarychev 2020,2022]) guarantees that, given two M-tuples of pairwise distinct points (M>1), we can steer one to the other. Moreover, this condition implies that we can approximate on compact sets any diffeomorphism isotopic to the identity using flows induced by the controlled dynamics.
We then formulate the (ensemble) optimal control problem related to the diffeomorphism approximation task, and we study its limiting behaviour when the size of the data-set tends to infinity. Finally, we show how this machinery can be used for the numerical construction of the optimal transport map.