TouchUp-G: Improving feature representation through graph-centric finetuning

How can we improve node features obtained from Pretrained Models (PMs) for downstream graph tasks? Graph Neural Networks (GNNs) have demonstrated promising results in various graph learning tasks, including node classification and link prediction. Despite their remarkable success in high-impact applications, we have observed that for feature-rich graphs, it is a common practice to directly employ a PM for feature generation in GNNs without incorporating any domain adaptation techniques. However, this practice is suboptimal because the node features extracted from PM are graph-agnostic and it prevents fully utilize the potential correlations between graph structures and node features. So how can we improve node features obtained from a PM for downstream graph tasks? We found that the best way is to do graph-centric finetuning on the PM.

In this paper, we present TouchUp-G; a simple Touch-Up enhancement technique to improve Graphs’ node features obtained from PMs via graph-centric pretraining. TouchUp-G has the following advantages: (a) General, can be applied to any downstream graph tasks; (b) Multi-modal, can improve raw features that come from any modality (e.g. images, texts); (c) Principled, we propose a novel metric: feature homophily to quantify the potential correlations between graph structures and node features; (d) Effective, outperforms baselines on 4 real datasets across various tasks and modalities, with up to 2× performance improvement on MRR.