ibmfl.party

Base Class

class ibmfl.party.training.local_training_handler.LocalTrainingHandler(fl_model, data_handler, hyperparams=None, **kwargs)[source]
eval_model(payload=None)[source]

Evaluate the local model based on the local test data.

Parameters

payload (dict) – data payload received from Aggregator

Returns

Dictionary of evaluation results

Return type

dict

save_model(payload=None)[source]

Save the local model.

Parameters

payload (dict) – data payload received from Aggregator

Returns

Status of save model request

Return type

boolean

sync_model(payload=None)[source]

Update the local model with global ModelUpdate received from the Aggregator.

Parameters

payload (dict) – data payload received from Aggregator

Returns

Status of sync model request

Return type

boolean

train(fit_params=None)[source]

Train locally using fl_model. At the end of training, a model_update with the new model information is generated and send through the connection.

Parameters

fit_params (dict) – (optional) Query instruction from aggregator

Returns

ModelUpdate

Return type

ModelUpdate

update_model(model_update)[source]

Update local model with model updates received from FusionHandler

Parameters

model_update (ModelUpdate) – ModelUpdate

Doc2Vec Local Training

class ibmfl.party.training.doc2vec_local_training_handler.Doc2VecLocalTrainingHandler ( fl_model, data_handler, hyperparams=None, **kwargs ) [source]

Class implementation for Doc2Vec Local Training Handler.

__init__ ( fl_model, data_handler, hyperparams=None, **kwargs ) [source]

Initialize LocalTrainingHandler with fl_model, data_handler

Parameters

  • fl_model (model.FLModel) – model to be trained

  • data_handler (DataHandler) – data handler that will be used to obtain data

  • hyperparams (dict) – Hyperparameters used for training.

  • kwargs (dict) – Additional arguments to initialize a local training handler.

Returns

None

update_model ( model_update ) [source]

Update local model with model updates received from FusionHandler

Parameters

  • model_update (ModelUpdate) – model update

Returns

None

train ( fit_params=None ) [source]

Primary wrapper function used for routing internal remote function calls within the Local Training Handler functions.

Parameters

  • fit_params (dict) – A dictionary payload structure containing two key signatures, `func` and `args`, which respectively are the target function defined within the Local Training Handler and the arguments defined within the executing function, which is defined as a dictionary containing key-value pairs of matching arguments.

Returns

the corresponding values depending on the function remotely called from the aggregator

eval_model ( payload=None ) [source]

Evaluate the local model based on data provided from the Aggregator

Parameters

  • payload (dict) – data payload received from Aggregator

Returns

list of evaluation results

Return type

list

get_vocabulary ( ) [source]

A remote function call which performs the following procedures: 1. Creates dictionary of words and their frequency in training data set

Parameters
Returns

Dictionary containing training dataset's vocabulary and frequencies

Return type

dict

set_initial_model ( set_initial_model ) [source]

Sets an initial Doc2Vec model received from the aggregator

Parameters

  • initial_model (gensim.models.doc2vec.Doc2Vec) – An initialized doc2vec model

Federated Averaging Local Training

class ibmfl.party.training.fedavg_local_training_handler.FedAvgLocalTrainingHandler(fl_model, data_handler, hyperparams=None, **kwargs)[source]
train(fit_params=None)[source]

Train locally using fl_model. At the end of training, a model_update with the new model information is generated and send through the connection.

Parameters

fit_params (dict) – (optional) Query instruction from aggregator

Returns

ModelUpdate

Return type

ModelUpdate

FedPlus Local Training

class ibmfl.party.training.fedplus_local_training_handler.FedPlusLocalTrainingHandler(fl_model, data_handler, hyperparams=None, **kwargs)[source]
update_model(model_update)[source]

Update local model with model updates received from FusionHandler.

Parameters

model_update (ModelUpdate)

Returns

None

sync_model(payload=None)[source]

Update the local model with global ModelUpdate received from the Aggregator.

Parameters

payload (dict)

Returns

Status of sync model request

Return type

boolean

soft_update_model(fit_params=None, key='weights'))[source]

Soft update to local model using global model and tuning alpha value.

Parameters

  • model_update (ModelUpdate)

  • key (str)

Returns

None

Gradient-Based Local Training

class ibmfl.party.training.gradient_local_training_handler.GradientLocalTrainingHandler(fl_model, data_handler, hyperparams=None, **kwargs)[source]

Class for stochastic gradient descent based fusion algorithms. In this class, the aggregation is performed over gradients.

train(fit_params)[source]

Retrieve the current gradient from party. With the provided model weights inside the fit_param, a model_weights including the current gradient information is generated and send through the connection.

Parameters

fit_params (dict) – Query instruction containing a set of model weights from aggregator

Returns

ModelUpdate

Return type

ModelUpdate

PFNM Local Training

class ibmfl.party.training.pfnm_local_training_handler.PFNMLocalTrainingHandler(fl_model, data_handler, hyperparams=None, **kwargs)[source]
train(fit_params=None)[source]

Train locally using fl_model. At the end of training, a model_update with the new model information is generated and send through the connection.

Parameters

fit_params (dict) – (optional) Query instruction from aggregator

Returns

ModelUpdate

Return type

ModelUpdate

Reweigh Local Training

class ibmfl.party.training.reweigh_local_training_handler.ReweighLocalTrainingHandler(fl_model, data_handler, hyperparams=None, **kwargs)[source]
__init__(fl_model, data_handler, hyperparams=None, **kwargs)[source]

Initialize LocalTrainingHandler with fl_model, data_handler.

Parameters

  • fl_model (model.FLModel)

  • data_handler (DataHandler)

  • hyperparams (dict)

  • kwargs (str) - Additional arguments to initialize a local training handler, e.g., a crypto library object to help with encryption and decryption.

Returns

None

train(fit_params=None)[source]

Train locally using fl_model. At the end of training, a model_update with the new model information is generated and send through the connection.

Parameters

  • fit_params (str) - (Optional) Query instruction from aggregator

Returns

ModelUpdate

Return type

ModelUpdate

eval_model(payload=None)[source]

Evaluate the local model based on the local test data.

Parameters

  • payload (dict) - data payload received from Aggregator

Returns

Dictionary of evaluation results

Return type

dict

Prejudice Remover Local Training

class ibmfl.party.training.pr_local_training_handler.PRLocalTrainingHandler(fl_model, data_handler, hyperparams=None, **kwargs)[source]
__init__(fl_model, data_handler, hyperparams=None, **kwargs)[source]

Initialize LocalTrainingHandler with fl_model, data_handler.

Parameters

  • fl_model (model.FLModel)

  • data_handler (DataHandler)

  • hyperparams (dict)

  • kwargs (str) - Additional arguments to initialize a local training handler, e.g., a crypto library object to help with encryption and decryption.

Returns

None

eval_model(payload=None)[source]

Evaluate the local model based on the local test data.

Parameters

  • payload (dict) - data payload received from Aggregator

Returns

Dictionary of evaluation results

Return type

dict

RL Local Training

Module to where local training handler are implemented.

class ibmfl.party.training.rl_local_training_handler.RLLocalTrainingHandler(fl_model, data_handler, hyperparams=None, **kwargs)[source]

Local training handler for RL

eval_model(payload=None)[source]

Evaluate the local model based on the local test data.

Parameters

payload (dict) – data payload received from Aggregator

Returns

Dictionary of evaluation results

Return type

dict

train(fit_params=None)[source]

Train locally using fl_model. At the end of training, a model_update with the new model information is generated and send through the connection.

Parameters

fit_params (dict) – (optional) Query instruction from aggregator

Returns

ModelUpdate

Return type

ModelUpdate

Party

class ibmfl.party.party.Party(**kwargs)[source]

Application that runs FL at the party side. Given a config file, it spins a server, creates FLModel of the type that will be trained, creates an instance of the DataHandler to be used.

evaluate_model()[source]

Calls function that evaluates current model with local testing data and prints the results.

Returns

None

register_party()[source]

Registers party

start()[source]

Start a server for the party in a new thread Parties can connect to the server to register

Param

None

Returns

New process to launch server

Return type

Process

stop()[source]

Stop the party server

Param

None

Returns

None

Party Protocol Handler

class ibmfl.party.party_protocol_handler.PartyProtocolHandler(fl_model, connection, data_handler, local_training_handler, hyperparams=None, agg_info=None, synch=False)[source]

Base class for all PartyProtocolHandlers

execute_async(id_request, msg)[source]

Handle run in a different thread to allow asynchronous requests.

Parameters

msg (Message) – Message object form connection

get_handle(message_type)[source]

Get handler for given message type.

Parameters

message_type (int) – request message type

Returns

a handler which was assigned for given message type

handle_async_request(msg)[source]
Handle all incoming requests asynchronously and route it to respective

methods in local training handler.

Parameters

msg (Message) – Message object form connection

Returns

Response message sent back to requester

Return type

ResponseMessage

handle_request(msg)[source]

Handle all incoming requests and route it to respective methods in local training handler.

Parameters

msg (Message) – Message object form connection

Returns

Response message sent back to requester

Return type

ResponseMessage

print_evaluate_local_model(hyperparams=None)[source]

Print local evaluations in console

Parameters

hyperparams (dict) – hyperparams for evaluation on the local model

Returns

None

Status Type

An enumeration class for the message type field which describe party status

class ibmfl.party.status_type.StatusType[source]

Status types for Party

Metrics Recorder

class ibmfl.party.metrics.metrics_recorder.MetricsEntry()[source]

Stores metrics for one round. The each round has values that can be added at the respective hook. The hooks should only fill their values and can use any keys they like (for now).

__init___()[source]

Just initializes a dictionary for each hook and creates a variable to keep track of which round this entry corresponds to.

Returns

None

to_dict()[source]

Converts the entry to a dictionary, simply mapping the variable names to keys.

Returns

None

class ibmfl.party.metrics.metrics_recorder.MetricsRecorder()[source]

Stores metrics for one round. The each round has values that can be added at the respective hook. The hooks should only fill their values and can use any keys they like (for now).

__init__()[source]

Just initializes a dictionary for each hook and creates a variable to keep track of which round this entry corresponds to.

Returns

None

add_entry()[source]

Create a new entry, i.e. for the upcoming round.

Returns

None

set_round_no()[source]

Update the most recent entry with the round number.

Returns

None

pre_train_hook()[source]

Fill in the pre_train dictionary of the latest entry.

Returns

None

post_train_hook()[source]

Fill in the post_train dictionary of the latest entry.

Returns

None

pre_update_hook()[source]

Fill in the pre_update dictionary of the latest entry; for now we just take timestamps.

Returns

None

post_update_hook()[source]

Fill in the post_update dictionary of the latest entry; for now we just take timestamps.

Returns

None

get_output_file()[source]

Return the formatted output file, including extension.

Returns

None

get_output_type()[source]

Return the extension (and hence type) of the output, just "json" for now.

Returns

None

to_json()[source]

Generate json for all of the metrics data currently stored.

Returns

None