Author
Pavan Balaji
Biography
Dr. Pavan Balaji is a Principal Research Scientist at Meta AI, where he serves as the technical lead for two areas: (1) GPU training systems (architectural design, performance analysis); and (2) AI
communication libraries for our various hardware systems (GPUs, Meta internal silicon). Dr. Balaji helped build some of Meta's largest AI supercomputing systems, such as the recent Grand Teton architecture, that power Meta's internal AI workloads, including recommendation and ranking models and Generative AI models such as Llama.
Before joining Meta, Dr. Balaji held appointments as a senior computer scientist and Group Lead at the Argonne National Laboratory and as an Institute Fellow of the Northwestern-Argonne Institute of Science and Engineering at Northwestern University. He contributed to the design and software implementation of a number of projects on communication runtime systems (MPI, UCX), threading models (lightweight threads such as Argobots, OpenMP), and heterogeneous memory systems. Particularly noteworthy are the MPICH project (used by thousands of supercomputers around the world, including the three U.S. exascale supercomputers —
Aurora, Frontier, and El Capitan), the UCX project (R&D100 Award winner in 2019), and the Argobots project (R&D100 Award finalist in 2020, and a driving piece of software for numerous supercomputers and commercial products such as Intel DAOS).
Dr. Balaji has held several other leadership roles in the community, serving on the board of directors or advisory board for numerous domestic and international projects, including UCX (U.S.), Cilk Plus (U.S.), EPEEC (Europe), and Exascale Technologies (China). He has also served on the organizing committees of numerous high-profile conferences and journals, including the IEEE/ACM SC Conference Series (Technical Program chair), IEEE Cluster (general co-chair), IEEE/ACM CCGrid (general co-chair, program chair), and IEEE TPDS (associate editor-in-chief).
communication libraries for our various hardware systems (GPUs, Meta internal silicon). Dr. Balaji helped build some of Meta's largest AI supercomputing systems, such as the recent Grand Teton architecture, that power Meta's internal AI workloads, including recommendation and ranking models and Generative AI models such as Llama.
Before joining Meta, Dr. Balaji held appointments as a senior computer scientist and Group Lead at the Argonne National Laboratory and as an Institute Fellow of the Northwestern-Argonne Institute of Science and Engineering at Northwestern University. He contributed to the design and software implementation of a number of projects on communication runtime systems (MPI, UCX), threading models (lightweight threads such as Argobots, OpenMP), and heterogeneous memory systems. Particularly noteworthy are the MPICH project (used by thousands of supercomputers around the world, including the three U.S. exascale supercomputers —
Aurora, Frontier, and El Capitan), the UCX project (R&D100 Award winner in 2019), and the Argobots project (R&D100 Award finalist in 2020, and a driving piece of software for numerous supercomputers and commercial products such as Intel DAOS).
Dr. Balaji has held several other leadership roles in the community, serving on the board of directors or advisory board for numerous domestic and international projects, including UCX (U.S.), Cilk Plus (U.S.), EPEEC (Europe), and Exascale Technologies (China). He has also served on the organizing committees of numerous high-profile conferences and journals, including the IEEE/ACM SC Conference Series (Technical Program chair), IEEE Cluster (general co-chair), IEEE/ACM CCGrid (general co-chair, program chair), and IEEE TPDS (associate editor-in-chief).
Presentations
Workshop
Artificial Intelligence/Machine Learning
Codesign
W
Tutorial
Parallel Programming Methods, Models, Languages and Environments
TUT
Paper
Artificial Intelligence/Machine Learning
Distributed Computing
Heterogeneous Computing
Performance Optimization
TP