-3

While I am studying Federated Learning, I have some questions that popped up in my mind that needed some clarification.

  1. We first have defined clients, each client will be split into training and testing sets. The training data are used to train the local models. Now, what testing data are used for? are they used to test the global model? or to test each local model?
  2. when training the global model, we first calculate the resulted weight of each local model, and then send it to the global model. In modeling the local clients, is there any validity check on the model itself before sending to the global model or it is sent anyway and then it will be updated by the global model.

Are there any papers explaining these points?

Eden
  • 325
  • 3
  • 13

1 Answers1

1
  1. Testing data is used to check your model accuracy. This can be useful for both local model and global model. However, since the objective of the federated learning is to build a unique global model, I would use the test data with the global model. There are, however, some approaches in which the local models'accuracy against a test set are used to give a weight to the local model before the "fusion" into the global model. This is sometimes reffered to as weighted FedAvG (federated averaging)
  2. In a "controlled" Federated Learning scenario, there is no reason to check each local model before being sent to the master. However, in realistic scenario, there are a lot of considerations regarding security that should be considered, therefore you might need something more robust than a simple "validity check"
Raoul Raftopoulos
  • 381
  • 1
  • 8
  • Thanks for your answer, I want some clarifications here. How could the test data used with global model since in federated learning setting, it should not be shared with the global model nor with other clients? I am still confused how the splitting is done. i.e in this [tutorial](https://www.tensorflow.org/federated/tutorials/federated_learning_for_image_classification) the data split in term of clients, meaning there are training and testing client. but how could the training in each client is done? – Eden Mar 19 '22 at 14:43
  • The testing data of the global model, in my opinion, would be "just another" split of the data that is available in the location in which the global model resides. Let's say for example that the global model runs in the cloud, whereas local models are trained in edge devices. It is feseable to think that each edge device will experience and build their datasets, whereas the cloud might have some other data to create its own testing dataset. This of course depends on the type of applications. – Raoul Raftopoulos Mar 21 '22 at 08:31
  • well, I kinda understand how it works in edge devices, but will it be the same with the smaller number of clients? e.g. in this [answer](https://stackoverflow.com/questions/71289273/tff-train-test-client-split-to-partition-each-client-data) the function `train_test_client_split` suggested split the clients into training and testing clients -since the data that I have are in different files each one is considered as one client. IDK if this is the right way to train clients? – Eden Mar 24 '22 at 01:02
  • With a smaller number of clients, I think you could try to have a test set only at the master node. Then, each client will train based on their training datasets, and send all the updates to the master node. After that, you update the global model and test its performance against its test set. This I think is better than having different test sets in each client, since usually the number of data in the clients is much lower w.r.t the ones in the master node. If, however, the master node doesn't have any test data, then you could try with the other clients test sets. Both approaches should work – Raoul Raftopoulos Mar 24 '22 at 09:03