publications
* denotes equal contribution
2024
- Geometric View of Soft Decorrelation in Self-Supervised LearningYifei Zhang, Hao Zhu, Zixing Song , Yankai Chen, Xinyu Fu, Ziqiao Meng, Piotr Koniusz, and Irwin KingIn KDD, 2024
Contrastive learning, a form of Self-Supervised Learning (SSL), typically consists of an alignment term and a regularization term. The alignment term minimizes the distance between the embeddings of a positive pair, while the regularization term prevents trivial solutions and expresses prior beliefs about the embeddings. As a widely used regularization technique, soft decorrelation has been employed by several non-contrastive SSL methods to avoid trivial solutions. While the decorrelation term is designed to address the issue of dimensional collapse, we find that it fails to achieve this goal theoretically and experimentally. Based on such a finding, we extend the soft decorrelation regularization to minimize the distance between the covariance matrix and an identity matrix. We provide a new perspective on the geometric distance between positive definite matrices to investigate why the soft decorrelation cannot efficiently solve the dimensional collapse. Furthermore, we construct a family of loss functions utilizing the Bregman Matrix Divergence (BMD), with the soft decorrelation representing a specific instance within this family. We prove that a loss function (LogDet) in this family can solve the issue of dimensional collapse. Our novel loss functions based on BMD exhibit superior performance compared to the soft decorrelation and other baseline techniques, as demonstrated by experimental results on graph and image datasets.
@inproceedings{DBLP:conf/kdd/ZhangZS00MKK24, author = {Zhang, Yifei and Zhu, Hao and Song, Zixing and Chen, Yankai and Fu, Xinyu and Meng, Ziqiao and Koniusz, Piotr and King, Irwin}, title = {Geometric View of Soft Decorrelation in Self-Supervised Learning}, booktitle = {{KDD}}, pages = {4338--4349}, publisher = {{ACM}}, year = {2024}, } - A Survey of Trustworthy Federated Learning: Issues, Solutions, and ChallengesACM Transactions on Intelligent Systems and Technology, 2024
Trustworthy artificial intelligence (TAI) has proven invaluable in curbing potential negative repercussions tied to AI applications. Within the TAI spectrum, federated learning (FL) emerges as a promising solution to safeguard personal information in distributed settings across a multitude of practical contexts. However, the realm of FL is not without its challenges. Especially worrisome are adversarial attacks targeting its algorithmic robustness and systemic confidentiality. Moreover, the presence of biases and opacity in prediction outcomes further complicates FL’s broader adoption. Consequently, there is a growing expectation for FL to instill trust. To address this, we chart out a comprehensive road-map for Trustworthy Federated Learning (TFL) and provide an overview of existing efforts across four pivotal dimensions: Privacy and Security, Robustness, Fairness, and Explainability. For each dimension, we identify potential pitfalls that might undermine TFL and present a curated selection of defensive strategies, enriched by a discourse on technical solutions tailored for TFL. Furthermore, we present potential challenges and future directions to be explored for in-depth TFL research with broader impacts.
@article{TrustworthyFL_survey, author = {Zhang, Yifei and Zeng, Dun and Luo, Jinglong and Fu, Xinyu and Chen, Guanzhong and Xu, Zenglin and King, Irwin}, title = {A Survey of Trustworthy Federated Learning: Issues, Solutions, and Challenges}, journal = {ACM Transactions on Intelligent Systems and Technology}, volume = {15}, number = {6}, pages = {112:1--112:47}, year = {2024}, } - A Systematic Survey on Federated Semi-supervised LearningIn IJCAI, 2024
Federated learning (FL) revolutionizes distributed machine learning by enabling devices to collaboratively learn a model while maintaining data privacy. However, FL usually faces a critical challenge with limited labeled data, making semi-supervised learning (SSL) crucial for utilizing abundant unlabeled data. The integration of SSL within the federated framework gives rise to federated semi-supervised learning (FSSL), a novel approach that exploits unlabeled data across devices without compromising privacy. This paper systematically explores FSSL, shedding light on its four basic problem settings that commonly appear in real-world scenarios. By examining the unique challenges, generic solutions, and representative methods tailored for each setting of FSSL, we aim to provide a cohesive overview of the current state of the art and pave the way for future research directions in this promising field.
@inproceedings{DBLP:conf/ijcai/SongYZ0XK24, author = {Song, Zixing and Yang, Xiangli and Zhang, Yifei and Fu, Xinyu and Xu, Zenglin and King, Irwin}, title = {A Systematic Survey on Federated Semi-supervised Learning}, booktitle = {{IJCAI}}, pages = {8244--8252}, publisher = {ijcai.org}, year = {2024}, } - MECCH: Metapath Context Convolution-based Heterogeneous Graph Neural NetworksXinyu Fu, and Irwin KingNeural Networks, 2024
Heterogeneous graph neural networks (HGNNs) were proposed for representation learning on structural data with multiple types of nodes and edges. To deal with the performance degradation issue when HGNNs become deep, researchers combine metapaths into HGNNs to associate nodes closely related in semantics but far apart in the graph. However, existing metapath-based models suffer from either information loss or high computation costs. To address these problems, we present a novel Metapath Context Convolution-based Heterogeneous Graph Neural Network (MECCH). MECCH leverages metapath contexts, a new kind of graph structure that facilitates lossless node information aggregation while avoiding any redundancy. Specifically, MECCH applies three novel components after feature preprocessing to extract comprehensive information from the input graph efficiently: (1) metapath context construction, (2) metapath context encoder, and (3) convolutional metapath fusion. Experiments on five real-world heterogeneous graph datasets for node classification and link prediction show that MECCH achieves superior prediction accuracy compared with state-of-the-art baselines with improved computational efficiency.
@article{DBLP:journals/nn/FuK24, author = {Fu, Xinyu and King, Irwin}, title = {{MECCH:} Metapath Context Convolution-based Heterogeneous Graph Neural Networks}, journal = {Neural Networks}, volume = {170}, pages = {266--275}, year = {2024}, }
2023
- FedHGN: A Federated Framework for Heterogeneous Graph Neural NetworksXinyu Fu, and Irwin KingIn IJCAI, 2023
Heterogeneous graph neural networks (HGNNs) can learn from typed and relational graph data more effectively than conventional GNNs. With larger parameter spaces, HGNNs may require more training data, which is often scarce in real-world applications due to privacy regulations (e.g., GDPR). Federated graph learning (FGL) enables multiple clients to train a GNN collaboratively without sharing their local data. However, existing FGL methods mainly focus on homogeneous GNNs or knowledge graph embeddings; few have considered heterogeneous graphs and HGNNs. In federated heterogeneous graph learning, clients may have private graph schemas. Conventional FL/FGL methods attempting to define a global HGNN model would violate schema privacy. To address these challenges, we propose FedHGN, a novel and general FGL framework for HGNNs. FedHGN adopts schema-weight decoupling to enable schema-agnostic knowledge sharing and employs coefficients alignment to stabilize the training process and improve HGNN performance. With better privacy preservation, FedHGN consistently outperforms local training and conventional FL methods on three widely adopted heterogeneous graph datasets with varying client numbers.
@inproceedings{DBLP:conf/ijcai/0004K23, author = {Fu, Xinyu and King, Irwin}, title = {FedHGN: {A} Federated Framework for Heterogeneous Graph Neural Networks}, booktitle = {{IJCAI}}, pages = {3705--3713}, publisher = {ijcai.org}, year = {2023}, }
2020
- MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph EmbeddingIn WWW, 2020
A large number of real-world graphs or networks are inherently heterogeneous, involving a diversity of node types and relation types. Heterogeneous graph embedding is to embed rich structural and semantic information of a heterogeneous graph into low-dimensional node representations. Existing models usually define multiple metapaths in a heterogeneous graph to capture the composite relations and guide neighbor selection. However, these models either omit node content features, discard intermediate nodes along the metapath, or only consider one metapath. To address these three limitations, we propose a new model named Metapath Aggregated Graph Neural Network (MAGNN) to boost the final performance. Specifically, MAGNN employs three major components, i.e., the node content transformation to encapsulate input node attributes, the intra-metapath aggregation to incorporate intermediate semantic nodes, and the inter-metapath aggregation to combine messages from multiple metapaths. Extensive experiments on three real-world heterogeneous graph datasets for node classification, node clustering, and link prediction show that MAGNN achieves more accurate prediction results than state-of-the-art baselines.
@inproceedings{DBLP:conf/www/0004ZMK20, author = {Fu, Xinyu and Zhang, Jiani and Meng, Ziqiao and King, Irwin}, title = {{MAGNN:} Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding}, booktitle = {{WWW}}, pages = {2331--2341}, publisher = {{ACM} / {IW3C2}}, year = {2020}, }
