ibmfl.util¶
Abstract Agent¶
-
class
ibmfl.util.custom_rl_core.agent.Agent(environment, hyperparams, memory)[source]¶ Abstract base class for all agents.
-
abstract
export_checkpoint(name)[source]¶ Export the model checkpoint for rollout
- Parameters
name – filename
-
abstract
get_action(state, *args)[source]¶ Returns the action to be taken given a state.
- Parameters
state – environment state in numpy array format
args – optional params
- Returns
action
-
abstract
get_weights() → {}[source]¶ Return model weights as dictionary, key is network id and value is weights in numpy array
- Returns
dictionary of model weights
-
abstract
load_state(name)[source]¶ Loads the agent state from the specified file
- Parameters
name – filename
-
abstract
save_state(name)[source]¶ Saves the agent state to the specified file
- Parameters
name – filename
-
abstract
Abstract Replay Memory¶
Abstract Rollout¶
DQN Agent¶
-
class
ibmfl.util.custom_rl_examples.cartpole_dqn.dqn_agent.DQNAgent(environment, hyperparams, memory)[source]¶ DQN Agent Implementation Taken from https://github.com/keon/deep-q-learning/blob/master/dqn.py
-
get_action(state)[source]¶ Returns the action to be taken given a state.
- Parameters
state – environment state in numpy array format
args – optional params
- Returns
action
-
get_weights() → {}[source]¶ Return model weights as dictionary, key is network id and value is weights in numpy array
- Returns
dictionary of model weights
-
DDPG Agent¶
-
class
ibmfl.util.custom_rl_examples.pendulum_ddpg.ddpg_agent.DDPGAgent(environment, hyperparams, memory)[source]¶ DDPG Agent Implementation Taken from https://gist.github.com/cyoon1729/542edc824bfa3396e9767e3b59183cae#file-ddpg-py
-
get_action(state)[source]¶ Returns the action to be taken given a state.
- Parameters
state – environment state in numpy array format
args – optional params
- Returns
action
-
get_weights() → {}[source]¶ Return model weights as dictionary, key is network id and value is weights in numpy array
- Returns
dictionary of model weights
-
parameters_to_numpy(params)[source]¶ Transforms parameter tensors to numpy arrays
- Parameters
params – The parameter tensor to be transformed
- Returns
numpy array of parameters
-
Critic Network¶
Actor Network¶
Ornstein-Ulhenbeck Process¶
Pendulum DDPG Rollout¶
Pendulum Replay Experience¶
Adult Decision Tree Data¶
-
class
ibmfl.util.data_handlers.adult_dt_data_handler.AdultDTDataHandler(data_config=None)[source]¶ Data handler for adult dataset.
-
get_data()[source]¶ Read adult.data from a given dir.
- Returns
A dataset structure
- Return type
pandas.core.frame.DataFrame
-
get_dataset_info()[source]¶ Read adult.data.txt and extract data information
- Returns
spec, a dictionary that contains list_of_features, feature_values and list_of_labels.
- Return type
dict
-
preprocess(training_dataset)[source]¶ Performs the following pre-processing on the provided dataset: * Categorize age feature into 4 categories,
i.e., [0, 18] -> ‘teen’, [18, 40] -> ‘adult’, [40, 80] -> ‘old-adult’, and [80, 99] -> ‘elder’.
- Categorize workclass feature into 3 categories,
i.e., [“?”, “Never-worked”, “Private”, “Without-pay”] -> ‘others’, [“Federal-gov”, ” Local-gov”] -> ‘gov’, and ->[“Self-emp-inc”, “Self-emp-not-inc”] -> ‘self’.
- Categorize education feature into 5 categories,
i.e., [“10th”,” 11th”, ” 12th”, ” 1st-4th”, ” 5th-6th”, ” 7th-8th”, ” 9th”] -> “non_college”, [” Assoc-acdm”, ” Assoc-voc”] ->”assoc”, [” Bachelors”, ” Some-college”] -> “college”, [” Doctorate”, ” HS-grad”, ” Masters”] -> “grad”, and [” Preschool”, ” Prof-school”] ->”others”
- Categorize education-num feature into 3 categories:
i.e., [0, 5] -> ‘<5’, [5, 10] -> ‘5-10’, and [10, 17] -> ‘>10’.
- Categorize capital-gain feature into 5 categories,
i.e., [-1, 1] -> 0, [1, 39999] -> 1,[39999, 49999] ->2, [49999, 79999] ->3, and [79999, 99999] ->4.
- Categorize capital-loss feature into 6 categories,
- i.e., [-1, 1] -0, [1, 999] ->1, [999, 1999] ->2, [1999, 2999] ->3,
[2999, 3999] ->4, [3999, 4499] -> 5.
- Categorize hours feature into 3 categories,
i.e., [0, 20] -> ‘<20’, [20, 40] -> ‘20-40’, [40, 100] -> ‘>40’.
- Categorize native-coutry into 5 categories,
i.e., [‘ ?’,] -> ‘others’, [‘ Cambodia’, ‘ China’, ‘ Hong’, ‘ India’, ‘ Iran’, ‘ Japan’, ‘ Laos’, ‘ Philippines’, ‘ South’, ‘ Taiwan’, ‘ Thailand’, ‘ Vietnam’] -> ‘asia’, [‘ Canada’, ‘ Outlying-US(Guam-USVI-etc)’, ‘ United-States’] -> ‘north_america’, [‘ Columbia’, ‘ Cuba’, ‘ Dominican-Republic’, ‘ Ecuador’, ‘ El-Salvador’, ‘ Guatemala’, ‘ Haiti’, ‘ Honduras’, ‘ Jamaica’, ‘ Mexico’, ‘ Nicaragua’, ‘ Peru’, ‘ Puerto-Rico’, ‘ Trinadad&Tobago’] -> ‘south_america’, [‘ England’, ‘ France’, ‘ Germany’, ‘ Greece’, ‘ Holand-Netherlands’, ‘ Hungary’, ‘ Ireland’, ‘ Italy’, ‘ Poland’, ‘ Portugal’, ‘ Scotland’, ‘ Yugoslavia’] -> ‘europe’.
Training label column is renamed to class
Function assumes dataset has column labels [‘1’, ‘2’, ‘3’…’13’, ‘class’], added in the load_adult() function in datasets.py, feature fnlwgt was also dropped in load_adult().
- Parameters
training_dataset (pandas.core.frame.DataFrame) – Raw training data
- Returns
Preprocessed training data
- Return type
pandas.core.frame.DataFrame
-
Adult Sklearn Data¶
-
class
ibmfl.util.data_handlers.adult_sklearn_data_handler.AdultSklearnDataHandler(data_config=None)[source]¶ Data handler for Adult dataset to train a Logistic Regression Classifier on scikit-learn. TEST_SIZE is set to 0.2, and RANDOM_STATE is set to 42.
-
get_col_names()[source]¶ Returns the names of the dataset columns
- Returns
column names
- Return type
list
-
get_data()[source]¶ Gets pre-processed adult training and testing data.
- Parameters
nb_points (int) – Number of data points to be included in each set
- Returns
training data
- Return type
tuple
-
get_weight()[source]¶ Gets pre-processed adult training and testing data, calculates weights for points weight = P-expected(sensitive_attribute & class)/P-observed(sensitive_attribute & class)
- Returns
weights
- Return type
np.array
-
preprocess(training_data)[source]¶ Performs the following preprocessing on adult training and testing data: * Drop following features: ‘workclass’, ‘fnlwgt’, ‘education’, ‘marital-status’, ‘occupation’,
‘relationship’, ‘capital-gain’, ‘capital-loss’, ‘hours-per-week’, ‘native-country’
- Map ‘race’, ‘sex’ and ‘class’ values to 0/1
‘ White’: 1, ‘ Amer-Indian-Eskimo’: 0, ‘ Asian-Pac-Islander’: 0, ‘ Black’: 0, ‘ Other’: 0
‘ Male’: 1, ‘ Female’: 0
Further details in Kamiran, F. and Calders, T. Data preprocessing techniques for classification without discrimination
Split ‘age’ and ‘education’ columns into multiple columns based on value
- Parameters
training_dataset (`pandas.core.frame.DataFrame) – Raw training data
- Returns
Preprocessed training data
- Return type
pandas.core.frame.DataFrame
-
Airline Data¶
Binovf Data¶
Diabetes Data¶
Federated Clustering Data¶
-
class
ibmfl.util.data_handlers.federated_clustering_data_handler.FederatedClusteringDataHandler(data_config=None)[source]¶ Data handler for a simulated federated clustering dataset.
-
get_data(**kwargs)[source]¶ Loads the simulated federated clustering dataset as described in the paper [https://arxiv.org/abs/1911.00218].
- :param **kwargs
See below
- Keyword Arguments
L (
int) – Number of true global centroids, default 100J (
int) – Number of clients, default 10D (
int) – Data dimension, default 50M (
int) – Data points per group, default 500mu0 (
float) – Global mean, default 0.0global_sd (
float) – Global standard deviation, default np.sqrt(L)local_sd (
float) – Local standard deviation, default 0.1
- Returns
(x_train, None), (x_test, None)
- Return type
tuple
-
Higgs Data¶
Linovf Data¶
MNIST Keras Data¶
-
class
ibmfl.util.data_handlers.mnist_keras_data_handler.MnistKerasDataHandler(data_config=None, channels_first=False)[source]¶ Data handler for MNIST dataset.
-
get_data(nb_points=500)[source]¶ Gets pre-process mnist training and testing data. Because this method is for testing it takes as input the number of datapoints, nb_points, to be included in the training and testing set.
- Parameters
nb_points (int) – Number of data points to be included in each set
- Returns
training data
- Return type
tuple
-
MNIST Sklearn Data¶
-
class
ibmfl.util.data_handlers.mnist_sklearn_data_handler.MnistSklearnDataHandler(data_config=None)[source]¶ Data handler for mnist dataset to train a SGDClassifier on scikit-learn.
-
get_data(nb_points=500)[source]¶ Gets pre-process mnist training and testing data. Because this method is for testing it takes as input the number of datapoints, nb_points, to be included in the training and testing set.
- Parameters
nb_points (int) – Number of data points to be included in each set
- Returns
training data
- Return type
tuple
-
Multovf Data¶
Multiclass Overfit Dataset Author: Yuya J. Ong (yuyajong@ibm.com)
Nursery Data¶
Pendulum Environment¶
-
class
ibmfl.util.data_handlers.pendulum_env.PendulumEnv(data=None, env_config=None, g=10.0)[source]¶ Github : https://github.com/openai/gym/blob/master/gym/envs/classic_control/pendulum.py Description:
Try to keep a frictionless pendulum standing up. The inverted pendulum swingup problem is a classic problem in the control literature. In this version of the
problem, the pendulum starts in a random position, and the goal is to swing it up so it stays upright
- Source:
Unknown
- Observation:
Type: Box(3) Num Observation Min Max 0 cos(theta) -1.0 1.0 1 sin(theta) -1.0 1.0 2 theta dot -8.0 8.0
- Actions:
Type: Box(1) Num Action Min Max 0 Joint effort -2.0 2.0
- Reward:
The precise equation for reward:
-(theta^2 + 0.1*theta_dt^2 + 0.001*action^2) Theta is normalized between -pi and pi. Therefore, the lowest reward is -(pi^2 + 0.1*8^2 + 0.001*2^2) = -16.2736044,
and the highest reward is 0. In essence, the goal is to remain at zero angle (vertical), with the least rotational velocity, and the least effort.
- Starting State:
Random angle from -pi to pi, and random velocity between -1 and 1
- Episode Termination:
There is no specified termination. Adding a maximum number of steps might be a good idea.
NOTE: Your environment object could be wrapped by the TimeLimit wrapper, if created using the “gym.make” method. In that case it will terminate after 200 steps.
-
close()[source]¶ Override close in your subclass to perform any necessary cleanup.
Environments will automatically close() themselves when garbage collected or when the program exits.
-
reset()[source]¶ Resets the state of the environment and returns an initial observation. Returns:
observation (object): the initial observation.
-
seed(seed=None)[source]¶ Sets the seed for this env’s random number generator(s).
- Note:
Some environments use multiple pseudorandom number generators. We want to capture all such seeds used in order to ensure that there aren’t accidental correlations between multiple generators.
- Returns:
- list<bigint>: Returns the list of seeds used in this env’s random
number generators. The first value in the list should be the “main” seed, or the value which a reproducer should pass to ‘seed’. Often, the main seed equals the provided ‘seed’, but this won’t be true if seed=None, for example.
-
step(u)[source]¶ Run one timestep of the environment’s dynamics. When end of episode is reached, you are responsible for calling reset() to reset this environment’s state. Accepts an action and returns a tuple (observation, reward, done, info). Args:
action (object): an action provided by the agent
- Returns:
observation (object): agent’s observation of the current environment reward (float) : amount of reward returned after previous action done (bool): whether the episode has ended, in which case further
step() calls will return undefined results
info (dict): contains auxiliary diagnostic information (helpful for debugging, and sometimes learning)
Pendulum Environment Data¶
Data Handler for Pendulum Environment
Rollout Save¶
-
class
ibmfl.util.data_handlers.rollout.RolloutSaver(outfile=None, use_shelve=False, write_update_file=False, target_steps=None, target_episodes=None, save_info=False)[source]¶ Utility class for storing rollouts. Currently supports two behaviours: the original, which simply dumps everything to a pickle file once complete, and a mode which stores each rollout as an entry in a Python shelf db file. The latter mode is more robust to memory problems or crashes part-way through the rollout generation. Each rollout is stored with a key based on the episode number (0-indexed), and the number of episodes is stored with the key “num_episodes”, so to load the shelf file, use something like: with shelve.open(‘rollouts.pkl’) as rollouts:
- for episode_index in range(rollouts[“num_episodes”]):
rollout = rollouts[str(episode_index)]
If outfile is None, this class does nothing.