I'm skeptical of the announcement from Ursula von der Leyen on the use of EU supercomputers by AI startups.
AFAIK they are not clusters of H100s / TPUs. It's not even close to be competitive infrastructure for training LLMs. LUMI for example runs on AMD MI250X GPUs.
Xavier Niel invests €200 million to create a French AI lab to be run by a nonprofit foundation, and purchasing an Nvidia DGX SuperPOD, equipped with 1016 H100s.
At least it will be more efficient than having to train LLMs on #LUMI ...
Interestingly, the new Mistral AI 7.3B parameter model was at least partially trained on CINECA's Leonardo cluster. Each node runs 4x NVIDIA AMPERE GPUs. Would be very curious to see the training costs on that publicly funded computation cluster.
"We thank the CINECA/EuroHPC team, and in particular the operators of Leonardo, for their resources and help"
Add comment