0

I am trying to implement that pysyft code for federated learing for my csv data . The tutorial i am following is this https://github.com/bt-s/Split-Learning-and-Federated-Learning/blob/master/src/federated_learning.py they used torch library FMNIST data which is iamge . I am having difficulty in customizing this code for my csv data.

This is error i am getting

File "C:/user/python/PCA/federated_learning.py", line 175, in train_loader = sy.FederatedDataLoader(train_set, transform=data_transformer.federate(workers), train=True, batch_size=args.batch_size, shuffle=True, **kwargs) AttributeError: 'Compose' object has no attribute 'federate

# Pysyft needs to be hooked to PyTorch to enable its features
    hook = sy.TorchHook(torch)

    # Define the workers
    alfa    = sy.VirtualWorker(hook, id="alfa")
    bravo   = sy.VirtualWorker(hook, id="bravo")
    workers = (alfa, bravo)

    device = "cuda" if torch.cuda.is_available() else "cpu"
    device = torch.device(device)
    kwargs = {'num_workers': 1, 'pin_memory': True} if device=="cuda" else {}

    # Specify required data transformation
    data_transformer = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize((0.5,), (0.5,))
    ])

    import pandas as pd
    print("Loading CSV...")
    test_set = pd.read_csv("C:/user/python/PCA/data/test.csv", encoding = "UTF-8")
    train_set = pd.read_csv("C:/user/python/PCA/data/train.csv", encoding = "UTF-8")
    train_loader = sy.FederatedDataLoader(train_set, transform=data_transformer.federate(workers), train=True, batch_size=args.batch_size, shuffle=True,  **kwargs)
    test_loader = torch.utils.data.DataLoader(test_set, transform=data_transformer, train=False, batch_size=args.batch_size, shuffle=True,  **kwargs)
user12
  • 761
  • 8
  • 24
  • This `data_transformer.federate(workers)` is the problem. The implementation [here](https://github.com/bt-s/Split-Learning-and-Federated-Learning/blob/master/src/federated_learning.py#L185) called `federate` over the `datasets.FashionMNIST` not the `Compose` – Natthaphon Hongcharoen Jul 03 '21 at 06:22
  • yeah i know this line is causing the problem but i am not able to resolve it – user12 Jul 03 '21 at 06:24
  • One way to make it runs is remove `.federate(workers)` but the question is why do you want it in the first place? Since I don't remember `torchvision.Dataset` has `.federate` method. – Natthaphon Hongcharoen Jul 03 '21 at 06:26
  • you can see in the tutorial link that they expalined workers and use .federate(workers) other way is to define every worker by name there .federate((alice, bob)) like in this tutorial https://www.kaggle.com/vedanthpadigelwar/federated-learning-and-privacy-preservation – user12 Jul 03 '21 at 06:28
  • it probably need torchvision 0.5.0 and I don't have that. – Natthaphon Hongcharoen Jul 03 '21 at 06:34
  • What I can suggest is, it has to be `train_set.federate(workers)` not the `data_transformer` – Natthaphon Hongcharoen Jul 03 '21 at 06:35
  • Both `transform=data_transforme` and `train=False` have to be in the `Dataset`. And since `pd.Dataframe` isn't a `Dataset` object so you'll have to write a new one. – Natthaphon Hongcharoen Jul 03 '21 at 06:37
  • already tried this one train_loader = sy.FederatedDataLoader(train_set.federate(workers), batch_size=args.batch_size, shuffle=True, **kwargs) it gives this error AttributeError: 'DataFrame' object has no attribute 'federate' – user12 Jul 03 '21 at 06:41
  • Yup, as I said `pd.DataFrame` isn't a `Dataset`. You have to write a custom dataset or maybe look for something like "pytorch pandas Dataset" – Natthaphon Hongcharoen Jul 03 '21 at 06:43
  • i didnt get you .. why i need to write custom dataset or need pytorch dataset i am reading my data from csv file – user12 Jul 03 '21 at 06:46

0 Answers0