Presentation

· Contributors · Organizations · Search Program · My Schedule · Happening Now · Maps

This content is available for: Tech Program Reg Pass. Upgrade Registration

TANGO: Re-Thinking Quantization for Graph Neural Network Training on GPUs

DescriptionGraph Neural Networks (GNNs) are rapidly gaining popularity since they hold state-of-the-art performance for various critical graph-related tasks. While quantization is a primary approach to accelerating GNN computation, quantized training faces remarkable challenges. We observe that current quantized GNN training systems often experience longer training time than their full-precision counterparts for two reasons: (i) addressing the accuracy challenge results in too much overhead. (ii) The optimization opportunity exposed by quantization is not well leveraged. This paper introduces Tango, which re-thinks quantization challenges and opportunities for graph neural network training on GPUs with the following contributions: First, we introduce light-weighted rules to meet the accuracy requirement for quantized GNN training. Second, we design and implement quantization-aware primitives and inter-primitive optimizations to accelerate GNN training. Third, we integrate Tango with the mainstream Deep Graph Library (DGL) system and demonstrate that Tango outperforms the state-of-the-art across all the evaluated GNN models and datasets.

Authors

University of Connecticut

Chengying Huan

Institute of Software, Chinese Academy of Sciences