ibmfl.aggregator

Comparative Elimination Fusion

class ibmfl.aggregator.fusion.comparative_elimination_fusion_handler.ComparativeEliminationFusionHandler ( hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs )

Class for Comparative Elimination (CE) Fusion.

Implements the Comparative Elimination algorithm presented here: https://arxiv.org/abs/2108.11769.

The CE fusion algorithm is a Byzantine-robust fusion algorithm. At high level, the CE fusion algorithm sorts the parties according to their l2-distance from the current global model (computed in the previous round), and selects the parties with smallest n - f diststances, where n is the number of parties and f is the estimated number of Byzantine parties.

__init__ ( hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs )

Initializes an ComparitiveEliminationFusionHandler object with provided information, such as hyperparams, protocol_handler, data_handler and fl_model.

Parameters

• hyperparams (dict) – Hyperparameters used for training

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm's request for communication

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – model to be trained

• kwargs (dict) – Additional arguments to initialize a fusion handler

fusion_collected_responses ( lst_model_updates, key='weights' )

Receives a list of model updates, where a model update is of the type ModelUpdate, and using the weights included in each model_update, it finds a robust aggregate model for the next round.

Parameters

• lst_model_updates (list) – List of model updates of type ModelUpdate

• key – A key indicating what values the method will aggregate over

Returns

Result after aggregation

Return type

list

get_distances_from_current_model ( lst_model_updates, key='weights' )

Computes distances between the current model and each model update.

Parameters

• lst_model_updates (list) – List of model updates participating in fusion round

• key – Key to pull from model update (default to ‘weights’)

Returns

distances

Return type

np.array

Coordinate-Median Aggregation Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.coordinate_median_fusion_handler.CoordinateMedianFusionHandler(hyperparams, protocol_handler, fl_model=None, data_handler=None, **kwargs)

Class for weight based Coordinate-Median aggregation.

In this class, the averaging aggregation is performed using Coordinate-Median policy model weights. Implements the algorithm in Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates: https://arxiv.org/pdf/1803.01498.pdf

__init__(hyperparams, protocol_handler, fl_model=None, data_handler=None, **kwargs)

Initializes an IterAvgFusionHandler object with provided information, such as protocol handler, fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – model to be trained

• kwargs (Dict) – Additional arguments to initialize a fusion handler.

Returns

None

fusion_collected_responses(lst_model_updates, key='weights')

Receives a list of model updates, where a model update is of the type ModelUpdate, using the values (indicating by the key) included in each model_update, it finds the update by combining the ModelUpdates together at each layer and determining the median of each layer

Parameters

• lst_model_updates (list) – List of model updates of type ModelUpdate to be averaged.

• key (str) – A key indicating what values the method will aggregate over.

Returns

results after aggregation

Return type

list

Doc2Vec Fusion

class ibmfl.aggregator.fusion.doc2vec_fusion_handler.Doc2VecFusionHandler ( hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs )

Class for fusion doc2vec models. Utilizes iterative averaging algorithm. An iterative fusion algorithm here refers to a fusion algorithm that sends out queries at each global round to registered parties for information, and use the collected information from parties to update the global model. The type of queries sent out at each round is the same. For example, at each round, the aggregator send out a query to request local model's weights after parties local training ends.

For doc2vec, the aggregator first requests a dictionary of all parties vocabulary and word frequency, and merges them before sending initial model Afterwards, the aggregator requests local model's weights from all parties at each round, and the averaging aggregation is performed over collected model weights. The global model's weights then are updated by the mean of all collected local models' weights.

__init__ ( hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs )

Initializes an Doc2VecFusionHandler object with provided information, such as protocol handler, fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – model to be trained

• kwargs (dict) – Additional arguments to initialize a fusion handler.

Returns

None

start_global_training ( )

Starts an iterative global federated learning training process.

evaluate_model ( data=None )

Requests all parties to send model evaluations.

Parameters

• data (str or TaggedDocument) – data to be evaluated by the registered parties' models

get_global_model ( )

Returns last model_update.

Returns

model_update

Return type

ModelUpdate

get_current_metrics ( )

Returns metrics pertaining to current state of fusion handler

Returns

metrics

Return type

dict

merge_vocab ( vocab_lists )

Combines vocabulary from dictionary of word frequencies

Parameters

• vocab_lists (list(dict)) – list of dictionaries containing the vocabulary words used in the training corpus

Returns

A dictionary with all words from each individual vocabulary dictionary

Return type

dict

set_initial_model ( merged_dict )

Sets an initial doc2vec model for parties to start with the same vocabulary and vector space

Parameters

• merged_dict (dict) – a dictionary containing all words and frequencies from all parties' training sets

Returns

an initialized doc2vec model

Return type

gensim.models.doc2vec.Doc2Vec

Decision Tree Fusion

class ibmfl.aggregator.fusion.dt_fusion_handler.ID3FusionHandler(hyperparams, proto_handler, data_handler, fl_model=None, **kwargs)

Class for training decision tree type model in aggregator side

__init__(hyperparams, proto_handler, data_handler, fl_model=None, **kwargs)

Initializes an DecisionTreeFusionHandler object with provided hyperparams, data_handler and fl_model.

Parameters

• hyperparams (dict) – Hyperparameters used for training

• proto_handler (ProtoHandler) – Proto_handler that will be used to send message

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – (optional) model to be trained

• kwargs (Dict) – Additional arguments to initialize a fusion handler.

build_branch(node, current_list_of_features=None, current_feature_values=None, splits=[])

Create a decision tree branch on a given node.

Parameters

• node (dict) – A given node to start building the tree

• current_list_of_features (list) – (Optional) A list stores current list of features that waiting to be split.

• current_feature_values (list) – (Optional) A list stores the corresponding feature value range.

• splits (list) – A list containing the tree split information, e.g. {[feature, feature_value]}

Returns

None

build_node(splits=[])

Create a tree node based on parties information, splits and max_depth requirement.

Parameters

splits (list) – A list containing the tree split information, e.g. {[feature_index, feature_value]}

Returns

A decision tree node

Return type

dict

fusion_collected_responses(lst_model_updates)

Receives a list of model updates, where a model update is of the type ModelUpdate, using the counts in each model_update, it returns the sum of all counts.

Parameters

of model updates (list) – Counts of type ModelUpdate to be summed up.

Returns

Model updates with sum of counts

Return type

ModelUpdate

get_global_model()

Returns latest tree model stored in fl_model object.

Returns

A dictionary contains the tree structure

Return type

ModelUpdate

reach_termination_criteria(root=None)

Return True when termination criteria has been reached, otherwise returns False. Termination criteria is reached when the tree grows to its leaves and there is nothing to be split.

Returns

boolean

Return type

‘boolean’

start_global_training(root=None)

Create a decision tree model.

Parameters

root (dict) – (Optional) the root of the decision tree

Returns

None

Federated Averaging Fusion

Module where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.fedavg_fusion_handler.FedAvgFusionHandler(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Class for weight based Federated Averaging aggregation.

Implements the FedAvg algorithm presented here: https://arxiv.org/pdf/1602.05629.pdf

__init__(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Initializes an FedAvgFusionHandler object with provided fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – Model to be trained

• kwargs (dict) – Additional arguments to initialize a fusion handler.

fusion_collected_responses(lst_model_updates)

Receives a list of model updates, where a model update is of the type ModelUpdate, using the weights included in each model_update, it finds the mean. It returns the weighted averaged model weights, where these weights to average model weights depends on parties’ sample size and indicating by key train_counts.

Parameters

lst_model_updates (list) – List of model updates of type ModelUpdate to be averaged.

Returns

weighted averaged model weights

Return type

list

Base Class

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.fusion_handler.FusionHandler(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Base class for Fusion

__init__(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Initializes an FusionHandler object with provided fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – model to be trained

• kwargs (dict) – Additional arguments to initialize a fusion handler.

__weakref__

list of weak references to the object (if defined)

evaluate_model()

Requests all parties to send model evaluations.

get_current_metrics()

Returns metrics pertaining to current state of fusion handler Includes all the the variables required to bring back fusion handler to the current state.

abstract get_global_model()

Returns the current global model at the aggregator side or model parameters that allow parties to reconstruct the global model.

get_registered_parties()

Returns a list of parties that registered for the current federated learning task.

Returns

lst_parties

Return type

list

initialization()

Perform initialization of the global training, e.g., warm-start setup etc.

Returns

None

query(function, payload, lst_parties=None, uniform_payload=True)

Generic query wrapper function to call arbitrary function defined within the local training handler of the party. Returns a list of the return values from each of the function, irrespective of whether they provide a return value or not.

Parameters

• function (str) – Name of function call that is defined within the local training handler

• payload (dict or list (of type dict)) – A dictionary which corresponds to the mapping of the necessary contents to pass into the argument provided in the function header. If uniform_payload is True, then distributes the same payload across all parties. If not, then each payload is distributed to each worker as defined by order present in the list of dictionaries.

• lst_parties (list) – List of parties to receive the query. Each entry of the list should be of type PartyConnection, and the length of the lst_parties should match the length of payload. If lst_parties is None, by default it will send queries to all parties as defined by get_registered_parties.

• uniform_payload (boolean) – A boolean indicator to determine whether the provided payload is the same across all parties. The default behavior is defined as distributing the same parameter across all parties.

Returns

response

Return type

list

query_all_parties(payload)

Sending queries to all registered parties. The query content is provided in payload.

Parameters

payload (dict) – Content of a query.

Returns

lst_model_updates: a list of replies gathered from the queried parties, each entry of the list should be of type ModelUpdate.

Return type

list

query_parties(payload, lst_parties)

Sending queries to the corresponding list of parties. The query contents is provided in payload. The corresponding recipients are provided in lst_parties.

Parameters

• payload (dict if a single query content will be sent to lst_parties or list if multiple queries will be sent to the corresponding parties specifying by lst_parties.) – Content of a query or contents of multiple queries

• lst_parties (list) – List of parties to receive the query. Each entry of the list should be of type PartyConnection, and the length of the lst_parties should match the length of payload if multiple queries will be sent.

Returns

lst_model_updates: a list of replies gathered from the queried parties, each entry of the list should be of type ModelUpdate.

Return type

list

save_current_state()

Save current fusion handler state using metrics manager. Save current model, collect metrics and use metricsmanager to save them.

save_local_model(filename=None)

Save aggregated model locally

save_parties_models()

Requests all parties to save local models.

send_global_model()

Send global model to all the parties

abstract start_global_training()

Starts global federated learning training process.

Fusion State Manager

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.fusion_state_service.FLFusionStateManager(object

Class for managing state of fusion.

__init__()

Keeps track of registered handlers, should be extented to handle event based handlers.

Parameters

None

Returns

None

register(handler)

Registers handlers/observers into the manager/scheduler

Parameters

• handler (method) – A routine which is invoked when scheduled.

Returns

None

save_state(state)

Registers handlers/observers into the manager/scheduler

Parameters

• state (method) – Invoke all the handlers registers for save event.

Returns

None

Fusion Util

class ibmfl.aggregator.fusion.fusion_handler.FusionUtil ( )

Base class for methods that can be used by fusion and local trainin algorithms.

static flatten_model_update ( lst_layerwise_wts )

Generates a flattened np array for all of the layerwise weights of an update

Parameters

• lst_layerwise_wts (list) – List of layer weights

Returns

array of layerwise weights

Return type

np.array

Gradient-Based Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.gradient_fusion_handler.GradientFusionHandler(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Class for gradient based aggregation and aggregated gradient descent

__init__(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Initializes an GradientFusionHandler object with provided information, such as protocol handler, fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – model to be trained

• kwargs (Dict) – Additional arguments to initialize a fusion handler.

Returns

None

update_weights(lst_model_updates)

Update the global model’s weights with the list of collected model_updates from parties. In this method, it calls self.fusion_collected_responses to average the gradients collected from parties, and performs a one-step gradient descent with learning rate as self.lr.

Parameters

lst_model_updates (lst) – List of model updates of type ModelUpdate to be averaged.

Returns

None

Iterative Averaging Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.iter_avg_fusion_handler.IterAvgFusionHandler(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Class for iterative averaging based fusion algorithms. An iterative fusion algorithm here referred to a fusion algorithm that sends out queries at each global round to registered parties for information, and use the collected information from parties to update the global model. The type of queries sent out at each round is the same. For example, at each round, the aggregator send out a query to request local model’s weights after parties local training ends. The iterative algorithms can be terminated at any global rounds.

In this class, the aggregator requests local model’s weights from all parties at each round, and the averaging aggregation is performed over collected model weights. The global model’s weights then are updated by the mean of all collected local models’ weights.

__init__(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Initializes an IterAvgFusionHandler object with provided information, such as protocol handler, fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – model to be trained

• kwargs (Dict) – Additional arguments to initialize a fusion handler.

Returns

None

fusion_collected_responses(lst_model_updates, key='weights')

Receives a list of model updates, where a model update is of the type ModelUpdate, using the values (indicating by the key) included in each model_update, it finds the mean.

Parameters

• lst_model_updates (list) – List of model updates of type ModelUpdate to be averaged.

• key (str) – A key indicating what values the method will aggregate over.

Returns

results after aggregation

Return type

list

get_current_metrics()

Returns metrics pertaining to current state of fusion handler

Returns

metrics

Return type

dict

get_global_model()

Returns last model_update

Returns

model_update

Return type

ModelUpdate

reach_termination_criteria(curr_round)

Returns True when termination criteria has been reached, otherwise returns False. Termination criteria is reached when the number of rounds run reaches the one provided as global rounds hyperparameter. If a DataHandler has been provided and a targeted accuracy has been given in the list of hyperparameters, early termination is verified.

Parameters

curr_round (int) – Number of global rounds that already run

Returns

boolean

Return type

boolean

start_global_training()

Starts an iterative global federated learning training process.

update_weights(lst_model_updates)

Update the global model’s weights with the list of collected model_updates from parties. In this method, it calls the self.fusion_collected_response to average the local model weights collected from parties and update the current global model weights by the results from self.fusion_collected_response.

Parameters

lst_model_updates (list) – list of model updates of type ModelUpdate to be averaged.

Returns

None

Krum Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.krum_fusion_handler.KrumFusionHandler(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Class for Krum Fusion.

Implements the Krum algorithm presented here: https://papers.nips.cc/paper/6617-machine-learning-with-adversaries-byzantine-tolerant-gradient-descent

__init__(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Initializes an KrumAvgFusionHandler object with provided fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – model to be trained

• kwargs (Dict) – Additional arguments to initialize a fusion handler.

fusion_collected_responses(lst_model_updates, key='weights')

Receives a list of model updates, where a model update is of the type ModelUpdate, using the weights included in each model_update, it finds the best model for the next round.

Parameters

• lst_model_updates (lst) – List of model updates of type ModelUpdate

• key – The key we wish to access from the model update

Returns

Result after fusion

Return type

list

get_distance(lst_model_updates, key)

Generates a matrix of distances between each of the model updates to all of the other model updates

Parameters

• lst_model_updates (list) – List of model updates participating in fusion round

• key – Key to pull from model update (default to ‘weights’)

Returns

distance

Return type

np.array

static get_scores(distance, th)

Sorts the distances in an ordered list and returns the list for use to the fusion_collected_responses function

Parameters

• distance (list) – List of distance vector

• th – Threshold

Returns

list of summation of distances

Return type

list

Gaussian Naive Bayes Fusion

class ibmfl.aggregator.fusion.naive_bayes_fusion_handler.NaiveBayesFusionHandler(hyperparams, proto_handler, data_handler, fl_model=None, **kwargs)

Class for Gaussian Naive Bayes federated learning with differential privacy.

Implements GaussianNB from diffprivlib, with party updates combined with the fusion handler.

__init__(hyperparams, proto_handler, data_handler, fl_model=None, **kwargs)

Initializes a NaiveBayesFusionHandler object with provided fl_model, data_handler, proto_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• proto_handler (ProtoHandler) – Proto_handler that will be used to send message

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – (optional) model to be trained

• kwargs (dict) – Additional arguments to initialize a fusion handler.

fusion_collected_responses(lst_model_updates, **kwargs)

Receives a list of model updates, where a model update is of the type ModelUpdate. Using the count, means and variances of each model_update, combines them into a single model update.

Parameters

lst_model_updates (list) – list of model updates of type ModelUpdate to be combined.

Returns

Model update with combined counts, means and variances.

Return type

ModelUpdate

get_global_model()

Returns last model_update

Returns

model_update

Return type

ModelUpdate

start_global_training()

Starts global federated learning training process.

PFNM Aggregation Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.pfnm_fusion_handler.PFNMFusionHandler(hyperparams, protocol_handler, fl_model, data_handler=None, **kwargs)

Class for weight based PFNM aggregation. The method is described here: https://arxiv.org/abs/1905.12022

This method supports only Fully-Connected Networks. Batch Normalization layer is not supported.

__init__(hyperparams, protocol_handler, fl_model, data_handler=None, **kwargs)

Initializes an PFNMFusionHandler object with provided fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for PFNM training. The four hyperparameters used are: sigma: float (default 1.0) Determines how far the local model neurons are allowed from the global model. A bigger value results in more matching and hence a smaller global model. sigma0: float (default 1.0) Defines the standard-deviation of the global network neurons. Acts as a regularizer. gamma: float (default 1.0) Indian Buffet Process parameter controlling the expected number of features present in each observation. iters: int (default 3) How many iterations of randomly initialized matching-unmatching procedure is to be performed before finalizing the solution

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• fl_model (model.FLModel) – model to be trained

• data_handler (DataHandler) – data handler that will be used to obtain data

• kwargs (Dict) – Additional arguments to initialize a fusion handler.

fusion_collected_responses(lst_model_updates, assignment_old=None)

Receives a list of model updates, where a model update is of the type ModelUpdate, using the weighs included in each model_update, it finds the mean. The averaged weights is stored in self.model_update as type ModelUpdate.

Parameters

• lst_model_updates (list) – List of model updates of type ModelUpdate to be averaged.

• assignment_old (np.ndarray) – Array of previous PFNM neuron assignments

Returns

None

get_current_metrics()

Returns metrics pertaining to current state of fusion handler

Returns

metrics

Return type

dict

get_global_model()

Returns last model_update

Returns

model_update

Return type

ModelUpdate

reach_termination_criteria(curr_round)

Returns True when termination criteria has been reached, otherwise returns False. Termination criteria is reached when the number of rounds run reaches the one provided as global rounds hyperparameter. If a DataHandler has been provided and a targeted accuracy has been given in the list of hyperparameters, early termination is verified.

Parameters

curr_round (int) – Number of global rounds that already run

Returns

boolean

Return type

boolean

start_global_training()

Starts global federated learning training process.

RL Averaging Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.rl_avg_fusion_handler.RLFusionHandler(hyperparams, protocol_handler, fl_model=None, data_handler=None, **kwargs)

Class for weight based Federated Averaging aggregation.

In this class, the simple averaging aggregation is performed over the RL policy model weights.

__init__(hyperparams, protocol_handler, fl_model=None, data_handler=None, **kwargs)

Initializes an IterAvgFusionHandler object with provided information, such as protocol handler, fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – model to be trained

• kwargs (Dict) – Additional arguments to initialize a fusion handler.

Returns

None

fusion_collected_responses(lst_model_updates)

Receives a list of model updates, where a model update is of the type ModelUpdate, using the weights included in each model_update, it finds the mean of weights per layer (indicating by key)

Parameters

lst_model_updates (lIst) – List of model updates of type ModelUpdate to be averaged.

Returns

results after aggregation

Return type

dict

Weight-Based RL Averaging Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.rl_weightedavg_fusion_handler.RLWeightedAvgFusionHandler(hyperparams, protocol_handler, fl_model=None, data_handler=None, **kwargs)

Class for weight based Federated Averaging aggregation.

In this class, the weighted averaging aggregation is performed over the RL policy model weights with averaging weights depends on rewards.

__init__(hyperparams, protocol_handler, fl_model=None, data_handler=None, **kwargs)

Initializes an FedAvgFusionHandler object with provided fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – Model to be trained

• kwargs (dict) – Additional arguments to initialize a fusion handler.

fusion_collected_responses(lst_model_updates)

Receives a list of model updates, where a model update is of the type ModelUpdate, using the weights and rewards included in each model_update, it finds the weighted average of the model weights per layer with averaging weights depends on rewards.

Parameters

lst_model_updates (list) – List of model updates of type ModelUpdate to be averaged.

Returns

results after aggregation

Return type

dict

SPAHM Aggregation Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.spahm_fusion_handler.SPAHMFusionHandler(hyperparams, protocol_handler, fl_model=None, data_handler=None, **kwargs)

Class for SPAHM aggregation of exponential family models. The method is described here: https://arxiv.org/abs/1911.00218

This method supports any model of the exponential family class

static __build_init__(hungarian_weights, assignments, j)

Create local weights from the matched global weights

Parameters

• hungarian_weights – Global network weights

• assignments – Assignment matrix mapping local to global neurons

• j – Network index for which updated local weights are required

Returns

local network weights

Return type

list of list

__init__(hyperparams, protocol_handler, fl_model=None, data_handler=None, **kwargs)

Initializes an SPAHMFusionHandler object with provided fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for SPAHM training. The five hyperparameters used are: 1. sigma: float (default 1.0) Determines how far the local model neurons are allowed from the global model. A bigger value results in more matching and hence a smaller global model. 2. sigma0: float (default 1.0) Defines the standard-deviation of the global network neurons. Acts as a regularizer. 3. gamma: float (default 1.0) Indian Buffet Process parameter controlling the expected number of features present in each observation. 4. iters: int (default 10) How many iterations of randomly initialized matching-unmatching procedure is to be performed before finalizing the solution 5. optimize_hyperparams: bool (default: True) Should SPAHM optimize the provided hyperparameters or not?

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• fl_model (model.FLModel) – model to be trained

• data_handler (DataHandler) – data handler that will be used to obtain data

• kwargs (`dict) – Additional arguments to initialize a fusion handler.

__prepare_payload__(lst_parties)

Prepares payload for each individual local model

Returns

payload for each client

Return type

list

static compute_cost(global_atoms, atoms_j, sigma, sigma0, mu0, popularity_counts, gamma, J)

Computes the full cost to be used by Hungarian algorithm. Refer equation (9) in the paper

fusion_collected_responses(lst_model_updates)

Receives a list of model updates, where a model update is of the type ModelUpdate, using the weighs included in each model_update, it finds the mean. The averaged weights is stored in self.model_update as type ModelUpdate.

Parameters

lst_model_updates (list) – List of model updates of type ModelUpdate to be averaged.

Returns

None

get_current_metrics()

Returns metrics pertaining to current state of fusion handler

Returns

metrics

Return type

dict

get_global_model()

Returns last model_update

Returns

model_update

Return type

ModelUpdate

static hyperparameters(global_atoms, global_atoms_squared, popularity_counts)

Estimates the hyperparameters mu0, sigma, and sigma0

match_local_atoms(local_atoms, sigma, sigma0, gamma, it, optimize_hyper=True)

Estimates the global atoms given the local atoms along with the hyperparameters.

matching_upd_j(atoms_j, global_atoms, global_atoms_squared, sigma, sigma0, mu0, popularity_counts, gamma, J)

Computes cost [Equation 9] and solves the linear assignment problem using hungarian algorithm

static objective(global_atoms, popularity_counts, sigma, sigma0, mu0)

Computes the full objective function

reach_termination_criteria(curr_round)

Returns True when termination criteria has been reached, otherwise returns False. Termination criteria is reached when the number of rounds run reaches the one provided as global rounds hyperparameter. If a DataHandler has been provided and a targeted accuracy has been given in the list of hyperparameters, early termination is verified.

Parameters

curr_round (int) – Number of global rounds that already run

Returns

boolean

Return type

boolean

start_global_training()

Starts global federated learning training process.

Zeno Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.zeno_gradient_fusion_handler.ZenoGradientFusionHandler(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Class for Zeno Fusion. Implements the Zeno algorithm presented here: http://proceedings.mlr.press/v97/xie19b.html

__init__(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Initializes an ZenoGradientFusionHandler object with provided information, such as protocol handler, fl_model, data_handler and hyperparams.

Parameters

• hyperparams (dict) – Hyperparameters used for training.

• protocol_handler (ProtoHandler) – Protocol handler used for handling learning algorithm’s request for communication.

• data_handler (DataHandler) – data handler that will be used to obtain data

• fl_model (model.FLModel) – model to be trained

Returns

None

fusion_collected_responses(lst_model_updates, key='gradients')

Receives a list of model updates and computes the score for each party as defined in Zeno

Parameters

• lst_model_updates (list) – List of model updates of type ModelUpdate to be averaged.

• key (str) – A key indicating what values the method will aggregate over.

Returns

results after aggregation

Return type

list

update_weights(lst_model_updates)

Update the global model’s weights with the list of collected model_updates from parties. In this method, it calls self.fusion_collected_responses to average the gradients collected from parties, and performs a one-step gradient descent with learning rate as self.lr.

Parameters

lst_model_updates (lst) – List of model updates of type ModelUpdate to be averaged.

Returns

None

Prejudice Remover Fusion

Module to where fusion algorithms are implemented.

class ibmfl.aggregator.fusion.prej_remover_fusion_handler.PrejudiceRemoverFusionHandler(hyperparams, protocol_handler, data_handler=None, fl_model=None, **kwargs)

Class for Prejudice Remover Fusion.

fusion_collected_responses(lst_model_updates, key)

Parameters

• lst_model_updates (list) – List of model updates of type `ModelUpdate` to be averaged.

• key (str) – A key indicating what values the method will aggregate over.

Returns

None

Reweigh Fusion

class ibmfl.aggregator.fusion.reweigh_fusion_handler.ReweighFusionHandler()

Class for iterative averaging based fusion algorithm with fairness application via the Reweighing Algorithm.

start_global_training()

Update the global model’s weights with the list of collected model_updates from parties. In this method, it calls the self.fusion_collected_response to average the local model weights collected from parties and update the current global model weights by the results from self.fusion_collected_response.

Parameters

None

Returns

None

global_reweighing(lst_replies)

Parameters

lst_replies (dict) – Party response with local DP counts for weight calculation.

Returns

global counts for weight calculation

Return type

dict

Protohandler Base Class

Module which will control overall Federated Learning algorithm such as plain FL, homomorphic encryption, Hybrid-one etc.

class ibmfl.aggregator.protohandler.proto_handler.ProtoHandler(connection, synch=False, max_timeout=None)

Base class for ProtoHandler (global federated learning algorithm)

__init__(connection, synch=False, max_timeout=None)

Initializes an ProtoHandler object

Parameters

• connection (Connection) – connection that will be used to send messages

• synch (boolean) – get model update synchronously

• max_timeout (int) – time in seconds to wait for parties

__weakref__

list of weak references to the object (if defined)

add_party(party)

Add a data party to federated learning process

Parameters

party (PartyConnection) – party details that are needed by the protocol handler

Return id

guid assigned to the party

eval_model_parties(party_ids, data)

Send eval request to parties by constructing a message with MessageType as EVAL_MODEL and packaging the data as payload

Parameters

• party_ids (list) – List of parties selected for an epoch

• data (dict) – query information which is sent to each party

Returns

message number

Return type

int

get_available_parties()

Returns all registered parties.

Returns

List of registered parties

Return type

list of PartyConnection

get_n_parties()

Return the number of parties as specified in the aggregator config file, which determines the legnth of parties_list

Returns

Number of parties that the aggregator expects to connect to

Return type

int

get_party_by_id(id)

Get Party_Connection object using id :param id: dictionary with information about source :type id: uuid

Returns

party

Return type

PartyConnection

get_party_by_info(info)

Get Party_Connection object using info received in message :param info: Dictionary with information about source :type info: dict

Returns

party

Return type

PartyConnection

has_reached_quorum(party_ids, id_request=None, id_request_list=None, perc_quorum=1.0)

Verifies if quorum has been reached. If it has, it returns True, otherwise it returns False.

Parameters

• party_ids (list) – List of parties that have received query

• id_request (int) – current id_request

• id_request_list (list) – A list of id number link to the query that needs quorum verification

• perc_quorum (float) – A float to specify percentage of parties that are needed for quorum to reach

Returns

Quorum status

Return type

boolean

periodically_verify_quorum(party_ids, id_request=None, id_request_list=None, perc_quorum=1.0)

Periodically verifies if enough replies from parties in party_ids for the current query identified by id_request have been received. If it has, returns True, and if it doesn’t in a maximum pre-defined time, it throws an exception

Parameters

• party_ids (list) – List of parties selected for an epoch

• id_request – An id number link to the query that needs quorum verification

• id_request_list (list) – A list of id number link to the query that needs quorum verification

• perc_quorum (float) – A float to specify percentage of parties that are needed for quorum to reach

Returns

boolean indicating if quorum has been reached

Return type

boolean

process_model_update_requests(message)

Save model update request send from party

Parameters

message (Message) – request send by party

Returns

Message with appropriate response

Return type

Message

query_parties(party_ids, data, msg_type=<MessageType.TRAIN: 7>)

Query a list of parties by constructing a message with MessageType default as TRAIN and packaging the data as payload.

Parameters

• party_ids (lst) – List of parties selected for an epoch

• data (dict) – query information which is sent to each party

• msg_type – The type of message the query should belong in. The default type is TRAIN, which allows the router to direct this type of queries to the train method defined inside the LocalTrainingHandler class. See MessageType class for other possible messages types.

Type

MessageType

Returns

message number

Return type

int

query_parties_data(party_ids, data_queries, msg_type=<MessageType.TRAIN: 7>)

Query a list of parties by constructing a message with MessageType as TRAIN and packaging the data as payload

Parameters

• party_ids (lst) – List of parties selected for an epoch

• data_queries (lst) – query information which is sent to each party

• msg_type – The type of message the query should belong in. The default type is TRAIN, which allows the router to direct this type of queries to the train method defined inside the LocalTrainingHandler class. See MessageType class for other possible messages types.

Type

MessageType

Returns

list of message numbers

Return type

list

quorum_failed_party(party_ids, id_request=None, id_request_list=None)

Verifies the parties that failed during quorum

Parameters

• party_ids (list) – List of parties that have received query

• id_request (int) – Current id_request

• id_request_list (list) – A list of id number link to the query that needs quorum verification

Returns

None

register_party(message)

Register data party :param message: Request received by the server :type message: Message

Returns

Message with success response

Return type

Message

save_model_parties(party_ids, data)

Send save model request to parties by constructing a message with MessageType as SAVE_MODEL and packaging the data as payload

Parameters

• party_ids (lst) – List of parties selected for an epoch

• data (dict) – query information which is sent to each party

Returns

message number

Return type

int

send_different_message_concurrently(party_ids, messages)

Send a message to list of parties asynchronously

Parameters

• party_ids (list) – list of parties to query

• messages (list) – List of Messages to be sent to party

Returns

Response message status

Return type

boolean

send_message(party_id, message)

Send a message to party

Parameters

• party (PartyConnection) – party to query

• message (Message) – Message to be sent to party

Returns

Response message status

Return type

boolean

stop_parties()

STOP all available parties.

Returns

message number

Return type

int

sync_model_parties(party_ids, data)

Send global model to a list of parties by constructinga message with MessageType as SYNC and packaging the model as payload

Parameters

• party_ids (lst) – List of parties selected for an epoch

• data (dict) – query information which is sent to each party

Returns

message number

Return type

int

Aggregator

Aggregator is an application which will allow users to execute controlled Federated Learning tasks

class ibmfl.aggregator.aggregator.Aggregator(**kwargs)

Aggregator class to create an aggregator application

__init__(**kwargs)

Initializes an Aggregator object

Parameters

config_file (str) – path to yaml file containing configuration

__weakref__

list of weak references to the object (if defined)

eval_model()

Request all parties to print evaluations

model_synch()

Send global model to the parties

save_model()

Request all parties to save models

start()

Start a server for the aggregator in a new thread Parties can connect to register

start_training()

Start federated learning training. Request all the registered parties to initiate training and send model update

Param

None

Returns

Boolean

Return type

boolean

stop()

Stop the aggregator server

Param

None

Returns

None

FL Metrics

class ibmfl.aggregator.metric_service.FLMetricsManager

__init__()

Keeps track of registered handlers, should be extented to handle event based handlers.

__weakref__

list of weak references to the object (if defined)

register(handler)

Registers handlers/observers into the manager/scheduler

Parameters

handler (method) – A routine which is invoked when scheduled

save_metrics(metrics)

Invoke all the handlers registers for save event and pass metrics

Parameters

metrics (dict) – Metrics dictionary

Party Connection

Module to track each party at the aggregator side

class ibmfl.aggregator.party_connection.PartyConnection(info)

Class for maintaining party related information for the ProtoHandler

__init__(info)

Initializes an PartyConnection object

__weakref__

list of weak references to the object (if defined)

add_new_reply(id_request, reply)

Adds the payload of the replied sent by the party in specific id_request to a dictionary where the key is the id_request and the reply the data.

Parameters

• id_request (int) – id_request for provided reply

• reply (may change) – payload associated with the reply message

Returns

None

get_party_metrics(id_request)

Returns party’s metrics for given id_request

Parameters

id_request (int) – training id_request

Returns

metric reply from given id_request

Return type

get_party_response(id_request)

Returns party’s response for given id_request

Parameters

id_request (int) – training id_request

Returns

reply from given id_request

Return type

has_party_replied(id_request)

Verifies if party has replied in provided id_request. If it does it returns True, otherwise returns false.

Parameters

id_request (int) – training id_request

Returns

if replied has been received

Return type

boolean

Aggregator States

An enumeration class for the message type field which describe Aggregator status

class ibmfl.aggregator.states.States

States for Aggregator