For optimal reading, please switch to desktop mode.
StackHPC's team attended the 21st ECMWF HPC Workshop, hosted at the
Tecnopolo cluster in Bologna, Italy. The Tecnopolo centre brings
together some of the most prominent Eurpoean and Italian research
institutions, including ECMWF, INFN, CINECA
and ENEA.
Happy 50th Birthday to ECMWF
ECMWF's 21st HPC workshop coincides with a significant 50th anniversary
milestone for the organisation, founded in 1975. The organisation
is established as a global leader in research into weather forecasting
and climate modelling.
Visualising and exploring weather model data in augmented reality
AI and Machine Learning Setting a New Agenda
Findings on early prototypes of ECMWF's AI-powered weather forecasting
system, AIFS,
were presented two years ago at ECMWF's 20th HPC workshop.
An opening keynote by Mariana Clare on recent work on AI-powered forecasting and modelling
Since then the AIFS model has quickly advanced in accuracy and
capability. It doesn't yet match the fine-grained resolution of the
numerical weather models. However, by many measures the forecast accuracy
of AIFS exceeds the numerical weather model.
Single and ensemble variants of the AIFS model are published on
huggingface. The
AIFS model can be used within a new python toolkit, Anemoi,
co-developed by ECMWF and a number of national meteorological
institutions. Anemoi is complemented by earthkit for supporting weather
and climate science workflows more widely.
ECMWF is now working on a pan-European Horizon2020 project,
WeatherGenerator to create a
foundation model trained on five substantial climate datasets, with
the objective of producing a model that is a digital twin of the
Earth as part of the Extreme Event and Climate programme of
Destination Earth.
EuroHPC and the AI Factories
The numerical weather prediction community has always been a community
that builds and uses giant machines, in order to model Earth's
weather systems with ever greater accuracy and detail. The HPC
systems are typically designed around the specific requirements of
one or more major applications.
The IT4LIA AI factory, hosted at CINECA, will integrate with CINECA's GAIA OpenStack cloud
Focus has shifted recently in the system requirements needed by the
weather and climate community, with increased focus on GPU support,
for both numerical weather models and AI models:
The JARVIS system planned at Jülich combines OpenStack, Kubernetes
and a technology stack familiar to StackHPC's clients working at
the convergence of HPC, AI and cloud.
The European Weather Cloud
StackHPC is proud to have implemented and to provide support for
ECMWF's component of the European Weather Cloud, a federation of OpenStack
infrastructures provided by ECMWF and EUMETSAT and harmonised to
provide a consistent user experience between both sites.
At the European Weather Cloud User Workshop it was a delight to hear
from some of the scientists and end users of the service about their
work made possible with the support of the Eurpoean Weather Cloud
service.
StackHPC's Poster Presentation
Finally, the StackHPC team were able to present a poster of the team's recent research into the combination of Slurm and Kubernetes, making opportunistic and preemptable use of valuable compute resources in a dynamic fashion. For more developments in this exciting area, watch this space!