Physics-inspired continuous learning models on graphs allow to overcome the limitations of traditional GNNs — The message-passing paradigm has been the “battle horse” of deep learning on graphs for several years, making graph neural networks a big success in a wide range of applications, from particle physics to protein design. From a theoretical viewpoint, it established the link to the Weisfeiler-Lehman hierarchy, allowing to analyse the expressive power of GNNs. I argue that the “node and edge-centric” mindset of current graph deep learning schemes imposes insurmountable limitations that obstruct future progress in the field. As an alternative, I propose physics-inspired “continuous” learning models that open up a new trove of tools from the fields of differential geometry, algebraic topology, and differential equations so far largely unexplored in graph ML.

Last year, I sought the opinion of leading researchers of Graph ML to make predictions about the future development in the field. This year, we teamed up with Petar Veličković and interviewed a cohort of distinguished and prolific experts in an attempt to summarise the highlights of the past year and predict what is in store for 2022. — This post was co-authored with Petar Veličković. See also my last year’s prediction, Michael Galkin’s excellent post on the current state of affairs in Graph ML, a deeper dive into subgraph GNNs, techniques inspired by PDEs and differential geometry and algebraic topology, and how the concepts of symmetry and invariance…

Most Graph Neural Networks typically run under the assumption of a full set of features available for all nodes. In real-world scenarios features are often only partially available (for example, in social networks, age and gender can be known only for a small subset of users). We show that Feature Propagation is an efficient and scalable approach for handling missing features in graph machine learning applications that works surprisingly well despite its simplicity.

The expressive power of Message-Passing Graph Neural Networks is inherently limited due to their equivalence to the Weisfeiler-Lehman graph isomorphism test. Several concurrent recent works show that this limitation can be overcome by applying a GNN on a collection of subgraphs obtained by removing nodes or edges from the input graph. In this post, we review the common themes and nuances of these different approaches.

Over-squashing is a common plight of Graph Neural Networks occurring when message passing fails to propagate information efficiently on the graph. In this post, we discuss how this phenomenon can be understood and remedied through the concept of Ricci curvature, borrowed from the field of differential geometry. — This post was co-authored with Francesco Di Giovanni and Jake Topping and is based on the paper J. Topping, F. Di Giovanni et al., Understanding over-squashing and bottlenecks on graphs via curvature (2021) arXiv:2111.14522, a collaboration between Twitter Cortex and the University of Oxford. This is the first installment of…

Differential geometry and algebraic topology are not encountered very frequently in mainstream machine learning. In this series of posts, I show how tools from these fields can be used to reinterpret Graph Neural Networks and address some of their common plights in a principled way. — “Symmetry, as wide or as narrow as you may define its meaning, is one idea by which man through the ages has tried to comprehend and create order, beauty, and perfection.”

Graph neural networks (GNNs) are intimately related to differential equations governing information diffusion on graphs. Thinking of GNNs as partial differential equations (PDEs) leads to a new broad class of GNNs that are able to address in a principled way some of the prominent issues of current Graph ML models such as depth, oversmoothing, bottlenecks, and graph rewiring. — This blog post was co-authored with Ben Chamberlain and James Rowbottom, and is based on our paper B. Chamberlain, J. Rowbottom et al., GRAND: Graph Neural Diffusion (2021) ICML.

Geometric Deep Learning is an attempt for geometric unification of a broad class of ML problems from the perspectives of symmetry and invariance. These principles not only underlie the breakthrough performance of convolutional neural networks and the recent success of graph neural networks but also provide a principled way to construct new types of problem-specific inductive biases. — This blog post was co-authored with Joan Bruna, Taco Cohen, and Petar Veličković and is based on the new “proto-book” M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković, Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges (2021), Petar’s talk at Cambridge and Michael’s keynote talk at ICLR…