Community-Aware Graph Transformers for Reducing Degree Bias in Node Representation Learning

Graphs offer a flexible platform to represent relational data in various fields including social networks, communication networks and biological networks and financial systems. Conventional graph representation learning algorithms, especially Graph Neural Networks (GNNs), are usually prone to degree bias, where high-degree nodes dominate the information-flowing process, resulting in unbalanced embedding’s and impaired generalization, and in heterogeneous networks in particular. To address this problem, we introduce a Community-Aware Graph Transformer (CGT) with the purpose of incorporating community structure information into the attention mechanism to improve the aggregation of information of nodes and alleviate the degree bias. The real-life network data, MIT Reality Mining and Enron Email, and Facebook Social networks were preprocessed and merged into one unified dataset with the extracted graph features, including node degree, clustering coefficient, and PageRank. Classical machine learning models (Logistic Regression, Random Forest, Gradient Boosting, SVM, KNN, Decision Tree) and deep learning models (DNN, RNN, LSTM, GRU, CNN1D, Bi-LSTM) were used to estimate the predictive performance of node embedding’s produced by CGT. Findings indicate that the deep learning models performed better, and the RNN-based models presented the best accuracy (99.97 percent), precision (99.90 percent), recall (99.87 percent), F1-Score (99.89 percent), and Cohen Kappa (99.95 percent). Classical ML models that are ensemble based like the Random Forest and Gradient Boosting also performed outstandingly as they reached 100% on all metrics whereas the simplistic models demonstrated slight constraints.