Tensorflow federated Issue with CSV dataset

Question

I ran the first 2 blocks of codes without any issue, I am trying to use my own dataset (network attacks similar to KDD) in a federated setup using Tensoflow_federated:

your text`csv_url = "/home/jupyter/tff.csv"
`your text`df = pd.read_csv(csv_url)
df = df.drop(columns=['Unnamed: 0'])
df = df.astype(float)
import random
client_id_colname = 'Client_id' # the column that represents client ID
SHUFFLE_BUFFER = 1000
NUM_EPOCHS = 1
number_of_training_clients = 2
# split client id into train and test clients
client_ids = df[client_id_colname].unique()
train_client_ids = random.sample(client_ids.tolist(),number_of_training_clients) 
test_client_ids = [x for x in client_ids if x not in train_client_ids]

#2
def create_tf_dataset_for_client_fn(client_id):
  client_data = df[df[client_id_colname] == client_id]
  dataset = tf.data.Dataset.from_tensor_slices(client_data.to_dict('list'))
  dataset = dataset.shuffle(SHUFFLE_BUFFER).batch(1).repeat(NUM_EPOCHS)
  return dataset`

However, I got this error when i tried to run that 3rd block, i tested it on another dataset and it was fine. the dataset is normalized:

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float).

import tensorflow_federated as tff
#Create a TFF FilePerUserClientData object for the training data
train_dataset_paths = {f"client_{i}": f"train_data_{i}.csv" for i in range(number_of_training_clients)}
for client_id, path in train_dataset_paths.items():
    client_data = df[df['Client_id'] == client_id]
    client_data.to_csv(path, index=False)
train = tff.simulation.datasets.FilePerUserClientData(
    train_dataset_paths,
    create_tf_dataset_for_client_fn
)

# Create a TFF FilePerUserClientData object for the testing data
test_dataset_paths = {f"client_{client_id}": f"test_data_{client_id}.csv" for client_id in test_client_ids}
for client_id, path in test_dataset_paths.items():
    client_data = df[df[client_id_colname] == client_id]
    client_data.to_csv(path, index=False)
test = tff.simulation.datasets.FilePerUserClientData(
    test_dataset_paths,
    create_tf_dataset_for_client_fn
)


`

I tested the code on another dataset and it worked just fine, and csv files were created for each client with its client ID.

Tensorflow federated Issue with CSV dataset

0 Answers0