#85 Dr. Petar Veličković (Deepmind) - Categories, Graphs, Reasoning [NEURIPS22 UNPLUGGED]

Theory Innovations

Dr. Petar Veličković, a leading researcher at DeepMind, is pioneering the use of category theory to enhance geometric deep learning models. He explains that category theory offers a framework to generalize geometric concepts beyond traditional symmetries, leading to new empirical findings that can be applied in graph neural networks 1. This approach allows researchers to take a "bird's eye view" of phenomena, making it easier to identify patterns across different fields 2.

Category theory is no more or no less than a way to take a bird's eye view of the phenomena that you try to study.

--- Petar Veličković

Tim Scarfe notes that this innovative application of category theory is not just theoretical but leads to practical advancements in model development.

   

Deep Learning

Category theory is transforming deep learning by providing a framework that extends beyond geometric symmetries. Petar Veličković highlights that while geometric deep learning focuses on equivariant layers, category theory allows for models that can handle non-invertible operations, such as scaling, which are common in natural and computational processes 3. This flexibility is crucial for developing models that remain effective across various domains.

You still might want to build a model that will give you the same answers regardless of how you scale up your input.

--- Petar Veličković

He also points out that category theory has been successfully applied in adjacent fields like quantum mechanics and ecosystems, demonstrating its broad applicability and potential to revolutionize how we construct neural networks 4.