Thoughts and Theory, Rethinking GNNs

Graph Neural Networks as Neural Diffusion PDEs

Graph neural networks (GNNs) are intimately related to differential equations governing information diffusion on graphs. Thinking of GNNs as partial differential equations (PDEs) leads to a new broad class of GNNs that are able to address in a principled way some of the prominent issues of current Graph ML models such as depth, oversmoothing, bottlenecks, and graph rewiring.

This blog post was co-authored with Ben Chamberlain and James Rowbottom, and is based on our paper B. Chamberlain, J. Rowbottom et al., GRAND: Graph Neural Diffusion (2021) ICML.

First page of “Scala graduum Caloris”, a 1701 paper by Sir Isaac Newton published anonymously in the Philosophical Transactions of the Royal Society. Shown is a temperature scale with 0 corresponding to the temperature of “winter air when water starts freezing” (aqua incipit gelu rigescere) and 12 representing the temperature measured upon “contact with the human body” (contactum corporis humani). The highest temperature of 210 is that of “kitchen fire urged by bellows”.

In March 1701, the Philosophical Transactions of the Royal Society published an anonymous note in Latin titled “A Scale of the Degrees of Heat” [1]. Though no name was indicated, it was no secret that Isaac Newton was the author (he would become “Sir Isaac” four years later). In a series of experiments, Newton observed that

the temperature a hot body loses in a given time is proportional to the…