Many RL problems have hierarchical tree-like nature. Hyperbolic geometry offers a powerful prior for such problems. — Many problems in Reinforcement Learning manifest a hierarchical tree-like nature. Hyperbolic spaces, which can be conceptualised as continuous analogies of trees, are thus suitable candidates to parameterise the agent’s deep model. In this post, we overview the basics of hyperbolic geometry, show empirically that it provides a good inductive bias…

Network games are a powerful tool for modelling strategic interactions between individuals or organisations played out on networks, where a player’s payoff depends not only on their own actions but also on those of their neighbours. Such games have numerous applications in economics and social sciences, including studying the spread of influence in social networks, the dynamics of financial markets, and the formation of alliances in international relations. The study of network games typically assumes the underlying network structure to be known, which is often wishful thinking. Recently, ML approaches have been proposed to tackle this problem by leveraging the observed actions of players to learn the underlying network structure. In this blog post, we outline a novel approach that uses a Transformer-like architecture to infer the network structure of a game without explicitly knowing the utility function associated with the game.

Geometric Deep Learning approaches a broad class of ML problems from the perspectives of symmetry and invariance, providing a common blueprint for the “zoo” of neural network architectures. In the last post in our series on the origins of Geometric Deep Learning, we look at the precursors of Graph Neural Networks. — In the last post from the “Towards Geometric Deep Learning” series, we discuss how early prototypes of GNNs emerged in the field of chemistry in the 1960s. This post is based on the introduction chapter of the book M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković, Geometric Deep…

Geometric Deep Learning approaches a broad class of ML problems from the perspectives of symmetry and invariance, providing a common blueprint for neural network architectures as diverse as CNNs, GNNs, and Transformers. In a new series of posts, we study how these ideas have taken us from ancient Greece to convolutional neural networks. — In the third post from the “Towards Geometric Deep Learning series,” we discuss the first “geometric” neural networks: the Neocognitron and CNNs. This post is based on the introduction chapter of the book M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković, Geometric Deep Learning (to appear with MIT…

Geometric Deep Learning approaches a broad class of ML problems from the perspectives of symmetry and invariance, providing a common blueprint for neural network architectures as diverse as CNNs, GNNs, and Transformers. In a new series of posts, we study how geometric ideas dating back to ancient Greece have shaped modern deep learning. — In the second post from the “Towards Geometric Deep Learning series,” we discuss the early neural network models, and how their criticism gave rise to a new field of computational geometry. This post is based on the introduction chapter of the book M. M. Bronstein, J. Bruna, T. Cohen, and…

Geometric Deep Learning approaches a broad class of ML problems from the perspectives of symmetry and invariance, providing a common blueprint for neural network architectures as diverse as CNNs, GNNs, and Transformers. In a new series of posts, we study how these ideas have emerged through history from ancient Greek geometry to Graph Neural Networks. — What is in common between snowflakes and the Standard Model? Symmetry. In the first post from the “Towards Geometric Deep Learning series,” we discuss how the concept of symmetry has helped organise the zoo of nineteenth-century geometries and revolutionise theoretical physics. This post is based on the introduction chapter of…

The suitability of standard hardware for Graph Neural Networks (GNNs) is an often overlooked issue in the Graph ML community. In this post, we explore the implementation of Temporal GNNs on a new hardware architecture developed by Graphcore that is tailored to graph-structured workloads. — This post was co-authored with Emanuele Rossi and Daniel Justus and is based on a collaboration with the British semiconductor company Graphcore.

Graph Neural Networks (GNNs) typically align their computation graph with the structure of the input graph. But are graphs the right computational fabric for GNNs? A recent line of papers challenges this assumption by replacing graphs with more general objects coming from the field of algebraic topology, which offer multiple theoretical and computational advantages. — This post was co-authored with Cristian Bodnar and Fabrizio Frasca and is based on the papers C. Bodnar, F. Frasca, et al., Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks (2021) ICML and C. Bodnar, F. Frasca et al., Weisfeiler and Lehman Go Cellular: CW Networks (2021) NeurIPS. It…

Graph Neural Networks (GNNs) are connected to diffusion equations that exchange information between the nodes of a graph. Being purely topological objects, graphs are implicitly assumed to have trivial geometry. Using a more general construction called cellular sheaves, we can endow the graph with a learnable geometric structure. We show that diffusion processes on cellular sheaves can solve any node classification task and can provide new insights about over-smoothing, the problem of dealing with heterophilic data, and the expressive power of GNNs.