Data and Model Geometry in Deep Learning

Abstract

Data with geometric structure is ubiquitous in machine learning. Often such structure arises from fundamental symmetries in the domain, such as permutation-invariance in graphs and sets, and translation-invariance in images. In this talk we discuss implications of this structure on the design and complexity of neural networks. Equivariant architectures, which encode symmetries as inductive bias, have shown great success in applications with geometric data, but can suffer from instabilities as their depths increases. We propose a new architecture based on unitary group convolutions, which allows for deeper networks with less instability. In the second part of the talk we discuss the impact of data and model geometry on the learnability of neural networks. We discuss learnability in several geometric settings, including equivariant neural networks, as well as learnability with respect to the geometry of the input data manifold.