How a wise use of direction when doing message passing on heterophilic graphs can result in very significant gains. — Graph Neural Networks (GNNs) are highly effective at modelling relational data. However, current GNN models frequently assume the input graph to be undirected, overlooking the inherent directionality of many real-world graphs, such as social, transportation, transaction, and citation networks. In this blog post, we explore the impact of edge directionality…

Under a few simple constraints, Graph Neural Networks can be derived as gradient flows minimising a learnable energy that describes attractive and repulsive forces in the feature space. This formalism allows the interpretation of GNNs as physical systems and sheds light onto how the interaction between the graph frequencies and the channel-mixing spectrum determine the evolution of the node features and control whether the dynamics is “smoothing” or “sharpening.” It also leads to a surprising conclusion: even very simple graph convolutional models with shared weights do not necessarily suffer from over-smoothing and can be efficient in heterophilic settings.

Many RL problems have hierarchical tree-like nature. Hyperbolic geometry offers a powerful prior for such problems. — Many problems in Reinforcement Learning manifest a hierarchical tree-like nature. Hyperbolic spaces, which can be conceptualised as continuous analogies of trees, are thus suitable candidates to parameterise the agent’s deep model. In this post, we overview the basics of hyperbolic geometry, show empirically that it provides a good inductive bias…

Network games are a powerful tool for modelling strategic interactions between individuals or organisations played out on networks, where a player’s payoff depends not only on their own actions but also on those of their neighbours. Such games have numerous applications in economics and social sciences, including studying the spread of influence in social networks, the dynamics of financial markets, and the formation of alliances in international relations. The study of network games typically assumes the underlying network structure to be known, which is often wishful thinking. Recently, ML approaches have been proposed to tackle this problem by leveraging the observed actions of players to learn the underlying network structure. In this blog post, we outline a novel approach that uses a Transformer-like architecture to infer the network structure of a game without explicitly knowing the utility function associated with the game.

Geometric Deep Learning approaches a broad class of ML problems from the perspectives of symmetry and invariance, providing a common blueprint for the “zoo” of neural network architectures. In the last post in our series on the origins of Geometric Deep Learning, we look at the precursors of Graph Neural Networks. — In the last post from the “Towards Geometric Deep Learning” series, we discuss how early prototypes of GNNs emerged in the field of chemistry in the 1960s. This post is based on the introduction chapter of the book M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković, Geometric Deep…

Geometric Deep Learning approaches a broad class of ML problems from the perspectives of symmetry and invariance, providing a common blueprint for neural network architectures as diverse as CNNs, GNNs, and Transformers. In a new series of posts, we study how these ideas have taken us from ancient Greece to convolutional neural networks. — In the third post from the “Towards Geometric Deep Learning series,” we discuss the first “geometric” neural networks: the Neocognitron and CNNs. This post is based on the introduction chapter of the book M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković, Geometric Deep Learning (to appear with MIT…

Geometric Deep Learning approaches a broad class of ML problems from the perspectives of symmetry and invariance, providing a common blueprint for neural network architectures as diverse as CNNs, GNNs, and Transformers. In a new series of posts, we study how geometric ideas dating back to ancient Greece have shaped modern deep learning. — In the second post from the “Towards Geometric Deep Learning series,” we discuss the early neural network models, and how their criticism gave rise to a new field of computational geometry. This post is based on the introduction chapter of the book M. M. Bronstein, J. Bruna, T. Cohen, and…

Geometric Deep Learning approaches a broad class of ML problems from the perspectives of symmetry and invariance, providing a common blueprint for neural network architectures as diverse as CNNs, GNNs, and Transformers. In a new series of posts, we study how these ideas have emerged through history from ancient Greek geometry to Graph Neural Networks. — What is in common between snowflakes and the Standard Model? Symmetry. In the first post from the “Towards Geometric Deep Learning series,” we discuss how the concept of symmetry has helped organise the zoo of nineteenth-century geometries and revolutionise theoretical physics. This post is based on the introduction chapter of…

The suitability of standard hardware for Graph Neural Networks (GNNs) is an often overlooked issue in the Graph ML community. In this post, we explore the implementation of Temporal GNNs on a new hardware architecture developed by Graphcore that is tailored to graph-structured workloads. — This post was co-authored with Emanuele Rossi and Daniel Justus and is based on a collaboration with the British semiconductor company Graphcore.