ugle 0.6.0

Description:

ugle 0.6.0

UGLE: Unsupervised GNN Learning Environment





Introduction
ugle is a library build on pytorch to compare implementations of GNNs for unsupervised clustering.
It consists of a wide range of methods, with the original implementations referenced in the source code.
We provide an experiment abstraction to compare different models and visualisation tools to plot results.
Any method can be trained individually via the main script using a specified config and the trainer objects.
Optuna is used to optimize hyperparameters, however models can be trained by specifying parameters.
Installation
To use this repository, you need to install pytorch_geometric so adjust the variables for the appropriate TORCH version and CUDA option. Please refer to the PyTorch Geometric installation guide if the following doesn't work for you as pytorch-geometric can be tricky with an M* Series Mac or Windows. Please use a virtual environment manager to install and use a version of python>=3.9.12. If you want a different version of pytorch then raise a pull request that can pass the version number to setup.py.

CUDA=cpu | cu102 | cu113 | cu116
pip install --extra-index-url https://data.pyg.org/whl/torch-1.12.0+{CUDA}.html ugle

Quick Tour
Here is an example of how you can use the python APIs from this repo.
import ugle
import numpy as np
n_nodes = 1000
n_features = 200
n_clusters = 3

# demo to evaluate a in Memory dataset
dataset = {'features': np.random.rand(n_nodes, n_features),
'adjacency': np.random.rand(n_nodes, n_nodes),
'label': np.random.randint(0, n_clusters+1, size=n_nodes)}

# load the dmon default hyperparameters and evaluate on in memory dataset
cfg = ugle.utils.load_model_config(override_model="dmon_default")
Trainer = ugle.trainer.ugleTrainer("dmon", cfg)
results = Trainer.eval(dataset)

# evalute dmon with hpo
Trainer = ugle.trainer.ugleTrainer("dmon")
Trainer.cfg.dataset = "cora"
Trainer.cfg.trainer.n_trials_hyperopt = 2 # this is how you change the config
Trainer.cfg.args.max_epoch = 250
results = Trainer.eval(dataset)

Changing the cfg property in the Trainer object will change the training and optimisation procedure as demonstrated above. Detailed below are all the options that can be changed.
ugle/configs/config.yaml
trainer:
gpu: "0" # GPU index (-1 = CPU)
show_config: False # prints the working config to cmd line

results_path: './results/' # where to save results to
data_path: './data/' # where to look for data
models_path: './models/' # where to save models

# if load_exising_test = True then trainer will look in {load_hps_path}{cfg.dataset}_{cfg.model}.pkl"
# for previous found hyperparameters
load_existing_test: False
load_hps_path: './found_hps/'

save_hpo_study: False # whether to save the hpo investigation
save_model: False # whether to save the best model

# hyperparameter options
test_metrics: ['f1', 'nmi', 'modularity', 'conductance'] # metrics to evaluate test data
valid_metrics: ['f1', 'nmi', 'modularity', 'conductance'] # metrics used for hpo and model selection
validate_every_nepochs: 5 # how many training epochs per validation
n_trials_hyperopt: 5 # how many hpo trials
max_n_pruned: 20 # patience for repeated hpo trials

training_to_testing_split: 0.2 # percentage of testing data compared to total of training+validation
train_to_valid_split: 0.2 # percentage of validation data as proportion of the whole dataset
split_scheme: 'drop_edges' # one of drop_edges, split_edges, all_edges, no_edges (see ugle.datasets.split_adj() for more info)

# logging options
calc_time: True # whether to calculate the time taken for evaluation
calc_memory: False # whether to calculate the memory used by evaluation
log_interval: 5 # how often to refresh the progress bar

args:
random_seed: 42 # the random seed to set
max_epoch: 5000 # the number of epochs to train for
# this is also where the specific model args are loaded into

If you want to do the same on the command line then you can parse use some of these arguments when running python main.py.
--model=<NAME-OF-MODEL>
--dataset=<NAME-OF-DATASET>
--seed=<SEED-ID>
--max_epoch=<MAX-EPOCHS-FOR-TRAINING>
--gpu=<GPU-INDEX>
--load_existing_test # if you specify a model as "dmon_custom" and write some custom args in the appropriate file that you want to test on then use this argument to load these

Running Experiment on Multiple (seeds/datasets/algorithms)
If you want to run a study over multiple seeds/dataset/algorithms then use the model_evaluations.py script alongside the experiments config files. See the ugle/configs/testing/ directory for examples on different scripts. If you have the null option in both datasets and algorithms in your config file then you can run the config over a single dataset and algorithm combination by using the -da argument which puts this combination into the dataset_algo_combinations option. This can also be used to define a list of combinations to choose to evaluate.
python3 model_evaluations.py -ec=<PATH-TO-EXP-CONFIG> -da=<DATASET>_<MODEL>
Existing GNN Implementations

daegc
dgi
dmon
mvgrl
grace
selfgnn
sublime
bgrl
vgaer
cagc

Existing Datasets

acm
amac
amap
bat
citeseer
cora
cocs
dblp
eat
uat
pubmed
texas
wisc
cornell
Physics
CS
Photo
Computers

how to add a new model <: MODEL-NAME>
To create a model with the name , you need to create minimum two files:

create a file for the model: ugle/models/<MODEL-NAME>.py
create a file to hold the hyperparameters: ugle/configs/models/<MODEL-NAME>/<MODEL-NAME>.yaml
optional* create file to hold default hyperparameters : ugle/configs/models/<MODEL-NAME>/<MODEL-NAME>_default.yaml
optional* create a new .yaml file to hold other variations: ugle/configs/models/<MODEL_NAME>/<MODEL_NAME>_myparameters.yaml
run an experiments or a single model run using the name reference as : "<MODEL_NAME>_myparameters"

Inside ugle/models/<MODEL-NAME>.py, define four hooks to process the whole model
from ugle.trainer import ugleTrainer # this is the base class for training any model in this framework

class <MODEL-NAME>_trainer(ugleTrainer):

def preprocess_data(self, features, adjacency):
# here we process the data into a required format
# some models using pytorch_geometric layers, some use custom layers
# this allows us to create new representation formats for features/adjacency matrix
# additional data structures needed can be returned for access under the tuple: processed_data

return (features, adj, ... )

def training_preprocessing(self, args, processed_data):
(features, adj, ...) = processed_data

# here the model and optimisers are defined as follows

self.model = ... Model()
self.optimizers = [optimizer]

return

def training_epoch_iter(self, args, processed_data):

# here is the training iterations hook, that defines the forward pass for each model
# the only requirement is that a loss is returned.
# If the processed data changes and is needed for the next iteration,
# then, return the tuple you need in place of processed data
# If this is not needed then None must be returned instead

return loss, (None/updated_processed_data)

def test(self, processed_data):

# the testing loop
# the definition of how predictions are returned by this model
# 1d numpy array returned of predictions for each node

return preds