Keypoints for Me
โข
Each graph generated by one sample can be viewed as sampling-based transformation
Critise
โข
Write parts that I cannot understand or fully agree with the paper
โข
Further used to look up other papers and might be erasable (erase with a strikethrough)
Minor Keypoints
โข
Write minor parts that I would like to investigate further (e.g. analysis/implementation tips)
Reference
Leave related page links in this page or a title with a link to arXiv
Summary
Motivation
โข
Contrastive Learning
โฆ
Contrastive Objectives
โฆ
Graph view generation
โช
Feature Transformation
โข
Node attribute masking : masking some node features or some nodes
โช
Structure Transformation
โข
Edge perturbation : masking edges! But it is not recommended for structured graphs
โข
Graph diffusion : Creating new connections using random walks.
โข
Centrality-based edge removal
โช
Sampling-based transformation
โข
Uniform sampling and node dropping : Uniformly sample nodes and create sub-graphs
โข
Ego-nets sampling : used to unify contrasts on graph-level and node-level. Use typical graph encoders to get node representation and use contrastive learning.
โฆ
Maybe similar to embedding images with different pretrained encoders?
โข
Random walk sampling : Sampling sub-graph using random walk.
โข
Network schema & Meta-path : Use individual ego-net for each node..?
โฆ
Predictive Learning
Training graph encoder together with a prediction head under supervision and self-supervision.
โช
Graph reconstruction
โข
Non-probabilistic Graph Autoencoders
โฆ
Reconstructing adjacency matrix
โช
GAE. GraphSAGE, SuperGAT, SimP-GCN
โช
MGAE, GALA,
โฆ
Attribute masking (graph completion) : reconstructing masked attributes
โข
Variational Graph Autoencoders
โฆ
VGAE, ARGA/ARVGA, SIG-VAE
โข
Autoregressive Reconstruction
โฆ
GPT, GPT-GNN
โช
Representation Invariance Regularization
Computing losses on representations similar to contrastive learning, but without pairs or negative samples.
Training model to minimize the difference between distorted graphs, leading them to seek distortion-invariant features
โข
BYOL, BGRL, CCA-SSGm LaGraph
Information Bottleneck principle
โช
Graph Property Prediction
Using predictive task based on graph properties not on graph data.
โข
S2GRL: k-hop connectivity prediction
โข
Meta-path prediction : Heterogeneous graph, finding 8 meta-paths
โข
GROVER
โฆ
Contextual property prediction : atom-bond-count on k-hop neighbor
โฆ
graph-level motif prediction : predict functional group in molecules.
โช
Self-training with Pseudo-labels
โข
Multi-stage self-training
โฆ
Train model with labeled dataset
โฆ
Get pseudo labels with high-confidence prediction on unlabeled dataset
โฆ
Trian new model on labeled datasetโฆ
โข
M3S - K-means clustering for pseudo labels
โข
ICF-GCN : iterative pseudo label refinement based on EM manner
Expansion assumption : correct pseudo-labels denoising incorrect pseudo-labels
โข
Unsupervised pretraining
โข
Auxiliary learning

