1

Higher-Order Learning with Graph Neural Networks via Hypergraph Encodings

Higher-order information is crucial for relational learning in many domains where relationships extend beyond pairwise interactions. …

R. Pellegrin, L. Fesser, M. Weber

Shared Global and Local Geometry of Language Model Embeddings

Researchers have recently suggested that models share common representations. In this work, we find that the token embeddings of …

A. Lee, M. Weber, F. Viegas, M. Wattenberg

Neural Feature Geometry Evolves as Discrete Ricci Flow

Deep neural networks learn feature representations via complex geometric transformations of the input data manifold. Despite the …

M. Hehl, M.-K. von Renesse, M. Weber

Position: Beyond Euclidean - Foundation Models Should Embrace Non-Euclidean Geometries

In the era of foundation models and Large Language Models (LLMs), Euclidean space has been the de facto geometric setting for machine …

N. He, J. Liu, B. Zhang, N. Bui, A. Maatouk, M. Yang, I. King, M. Weber, R. Ying

Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups

The quest for robust and generalizable machine learning models has driven recent interest in exploiting symmetries through equivariant …

Z. Shumaylov, P. Zaika, J. Rowbottom, F. Sherry, M. Weber, C.-B. Schönlieb

Performance Heterogeneity in Graph Neural Networks: Lessons for Architecture Design and Preprocessing

Graph Neural Networks have emerged as the most popular architecture for graph-level learning, including graph classification and …

L. Fesser, M. Weber

Unitary convolutions for learning on graphs and groups

Group-convolutional architectures, which encode symmetries as inductive bias, have shown great success in applications, but can suffer …

B. T. Kiani, L. Fesser, M. Weber

Hardness of Learning Neural Networks under the Manifold Hypothesis

The manifold hypothesis presumes that high-dimensional data lies on or near a low-dimensional manifold. While the utility of encoding …

B. T. Kiani, J. Wang, M. Weber

Effective Structural Encodings via Local Curvature Profiles

Structural and Positional Encodings can significantly improve the performance of Graph Neural Networks in downstream tasks. Recent …

L. Fesser, M. Weber

On the Hardness of Learning under Symmetries

We study the problem of learning equivariant neural networks via gradient descent. A recent line of learning theoretic research has …

B. T. Kiani, T. Le, H. Lawrence, S. Jegelka, M. Weber