Johannes Brandtstetter, Professor at Johannes Kepler University in Linz and Research Scientist at Microsoft Research

Leuchs-Russell-Auditorium, A.1.500, Staudtstr. 2

Location details

Abstract:

In the era of GPT models, one gets notoriously confronted with the question of how much inductive (geometric) bias can help when scaling up deep neural architectures. In this talk, we contrast contemporary geometric deep learning approaches, such as group equivariant learning, with large-scale Transformer success stories. The discussion narrows its focus to recent triumphs in weather and climate modeling within the realm of Transformers, and subsequently extrapolates insights applicable to diverse scientific and engineering fields. Most notably, we outline ongoing developments in the modeling of large-scale systems such as fluid dynamics and the respective impact on industry and scientific environments.