ACM, the Affiliation for Computing Equipment, immediately named Torsten Hoefler, a professor at ETH Zurich, the recipient of the ACM Prize in Computing for basic contributions to high-performance computing and the continued AI revolution.
Hoefler, who was a visitor on the @HPCpodcast on the location of this publication’s sister outlet, insideHPC, final November, developed most of the core capabilities of recent supercomputers and outlined key features of the algorithms for distributing AI fashions on them, the ACM stated.
The ACM Prize in Computing acknowledges early-to-mid-career laptop scientists whose analysis contributions have basic influence and broad implications. The award carries a prize of $250,000, from an endowment supplied by Infosys Ltd., a digital companies and consulting firm.
ACM cited the work of Hoefler and his colleagues to scale community design and programming in supercomputers, which has revolutionized the capabilities of those
massive programs. For instance, AI algorithms can now be processed on a whole bunch of hundreds of nodes. Hoefler’s advances in interconnection networks, programming, and parallel algorithms broke new floor in facilitating using large-scale massively parallel clusters.
“His quite a few improvements throughout the entire supercomputer stack—together with key contributions equivalent to MPI-3 nonblocking collective operations, foundational parallelism methods for AI fashions, and high-performance networking programs—have pushed the boundaries of parallel programs design and translated into dramatic enhancements in supercomputer efficiency and scalability,” the ACM stated. “A lot of these improvements are integrated into the most important and strongest machines immediately.”
The ACM cited these examples of Hoefler’s HPC-AI R&D:
Message Passing Interface 3: Hoefler performed a serious function within the evolution of the Message Passing Interface (MPI), an off-the-cuff trade customary for exchanging messages between quite a few particular person nodes all through an HPC community. A messaging customary permits synchronization of the actions of every particular person laptop, sharing information between nodes, and route and management of your entire parallel community.
The MPI-3 customary was adopted in 2012 and made potential advances in HPC for simulations and AI functions over the previous a number of years. Hoefler chaired each the “Course of Topologies” and “Collective Operations” working teams for MPI-3. His nonblocking collective operations, equivalent to Allreduce, Allgather, Bcast, and their respective blocking variations are included in numerous collective communication libraries—even past MPI-3. These operations energy the core of distributed deep studying immediately.
3D Parallelism: Hoefler was among the many first to develop and uncover the now well-known notion of “3D parallelism,” which drives infrastructure design for the AI trade. Subsequently, he and his collaborators continued to develop strategies for environment friendly pipelining, sparse communication, mannequin sparsity and quantization. This work has
enabled a cumulative 10-1000x acceleration of AI workloads in fashionable computer systems.
Routing Protocols and Community Topologies: The low-level community routing protocols and community topologies that Hoefler and his colleagues developed for such networks as Myrinet and InfiniBand energy hundreds of AI and HPC supercomputers. These contributions are central to high-performance AI programs that prepare large-language fashions, equivalent to ChatGPT.
“The capability of high-performance computer systems has grow to be mind-boggling,” stated ACM President Yannis Ioannidis. “In simply the final two years, we’ve got ushered within the period of
exascale computer systems, which might carry out a billion billion calculations per second. However high-performance computer systems might do little with out the underlying algorithms and requirements that enable them to course of large influxes of knowledge.
“Whereas Hoefler launched many of those improvements whereas he was nonetheless a scholar, they continue to be the definitive option to program massively parallel programs immediately,” Ioannidis continued. “The ensuing capabilities of those programs have allowed important advances in AI, the pure sciences, and plenty of different areas.”
Salil Parekh, CEO, Infosys, stated, “We’re in the course of an thrilling period of AI, with nice promise for the long run. Torsten Hoefler performed an necessary function in advancing high-performance computing, which in flip, fuelled the present AI revolution. The ACM Prize in Computing is meant to acknowledge to ‘early to mid-career professionals,’ whose work has depth and influence. With a lot of his necessary work being executed in his 20s, Hoefler is an instance for younger folks that age just isn’t an impediment to achievement in computing. Infosys is proud to be the sponsor of this award because it was based in 2007.”
Hoefler is a Professor of Laptop Science at ETH Zurich (the Swiss Federal Institute of Expertise), the place he serves as Director of the Scalable Parallel Computing
Laboratory. He’s additionally the Chief Architect for AI and Machine Studying on the Swiss Nationwide Supercomputing Centre (CSCS). Hoefler acquired a Diplom Informatik (Grasp of Laptop Science) from Chemnitz College of Expertise and a PhD in Laptop Science from Indiana College.
Hoefler’s honors embody the Max Planck-Humboldt Medal, an award for excellent mid- profession scientists; the IEEE CS Sidney Fernbach Award, which acknowledges excellent contributions within the utility of high-performance computer systems; and the ACM Gordon Bell Prize, which acknowledges excellent achievement in high-performance computing. He’s a member of the European Academy of Sciences (Academia Europaea), a Fellow of IEEE, and a Fellow of ACM.
Hoefler can be formally offered with the ACM Prize in Computing at ACM’s annual awards banquet on Saturday, June 14 at The Palace Resort in San Francisco.
