1

Shared Global and Local Geometry of Language Model Embeddings

Researchers have recently suggested that models share common representations. In this work, we find that the token embeddings of …

A. Lee, M. Weber, F. Viégas, M. Wattenberg

Lie algebra canonicalization: Equivariant neural operators under arbitrary Lie groups

The quest for robust and generalizable machine learning models has driven recent interest in exploiting symmetries through equivariant …

Z. Shumaylov*, P. Zaika*, J. Rowbottom, F. Sherry, M. Weber, C.-B. Schönlieb

Performance Heterogeneity in Graph Neural Networks: Lessons for Architecture Design and Preprocessing

Graph Neural Networks have emerged as the most popular architecture for graph-level learning, including graph classification and …

L. Fesser, M. Weber

Unitary convolutions for learning on graphs and groups

Group-convolutional architectures, which encode symmetries as inductive bias, have shown great success in applications, but can suffer …

B. T. Kiani, L. Fesser, M. Weber

Hardness of Learning Neural Networks under the Manifold Hypothesis

The manifold hypothesis presumes that high-dimensional data lies on or near a low-dimensional manifold. While the utility of encoding …

B. T. Kiani, J. Wang, M. Weber

Effective Structural Encodings via Local Curvature Profiles

Structural and Positional Encodings can significantly improve the performance of Graph Neural Networks in downstream tasks. Recent …

L. Fesser, M. Weber

On the Hardness of Learning under Symmetries

We study the problem of learning equivariant neural networks via gradient descent. A recent line of learning theoretic research has …

B. T. Kiani, T. Le, H. Lawrence, S. Jegelka, M. Weber

Mitigating Over-Smoothing and Over-Squashing using Augmentations of Forman-Ricci Curvature

While Graph Neural Networks (GNNs) have been successfully leveraged for learning on graph-structured data across domains, several …

L. Fesser, M. Weber

Sampling Informative Positives Pairs in Contrastive Learning

Contrastive Learning is a paradigm for learning representation functions that recover useful similarity structure in a dataset based on …

M. Weber, P. Bachman

Global optimality for Euclidean CCCP under Riemannian convexity

We study geodesically convex problems that can be written as a difference of Euclidean convex functions. This structure arises in key …

M. Weber, S. Sra