Pruning graph neural networks by evaluating edge properties

No Thumbnail Available
File version
Author(s)
Wang, Li
Huang, Wei
Zhang, Miao
Pan, Shirui
Chang, Xiaojun
Su, Steven Weidong
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
2022
Size
File type(s)
Location
License
Abstract

The emergence of larger and deeper graph neural networks (GNNs) makes their training and inference increasingly expensive. Existing GNN pruning methods simultaneously prune the graph adjacency matrix and the model weights on a pretrained neural network by directly leveraging the lottery-ticket hypothesis, but the benefits of such methods are mainly via weight pruning, and methods based on saliency metrics struggle to outperform random pruning when pruning only the graph adjacency matrix. This motivates us to use different scoring standards for graph edges and network weights during GNN pruning. Thus, rather than measuring the importance of graph edges based on saliency metrics, we formulate the performance of GNNs mathematically with respect to the properties of their edges, elucidating how the performance drop can be avoided by pruning negative edges and nonbridges. This leads to our simple but effective two-step method for GNN pruning, leveraging the saliency metrics for the network pruning while sparsifying the graph with preservation of the loss performance. Experimental results show the effectiveness and efficiency of the proposed method on both small-scale graph datasets (Cora, Citeseer, and PubMed) and a large-scale dataset (Ogbn-ArXiv), where our method saves up to 98% of floating-point operations per second (FLOPs) on the small graphs and 94% of FLOPs on the large one, with no significant drop in accuracy.

Journal Title

Knowledge-Based Systems

Conference Title
Book Title
Edition
Volume

256

Issue
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
Item Access Status
Note
Access the data
Related item(s)
Subject

Knowledge representation and reasoning

Neural networks

Science & Technology

Technology

Computer Science, Artificial Intelligence

Computer Science

Graph neural networks

Persistent link to this record
Citation

Wang, L; Huang, W; Zhang, M; Pan, S; Chang, X; Su, SW, Pruning graph neural networks by evaluating edge properties, Knowledge-Based Systems, 2022, 256, p. 109847

Collections