1

I want to print local outputs of clients in the tensorflow federated tutorial https://www.tensorflow.org/federated/tutorials/federated_learning_for_image_classification. What should I do?

ester
  • 11
  • 2

2 Answers2

1

If you only want a list of the values that go into the aggregations (e.g. into tff.federated_mean), one option would be to add additional outputs to aggregate_mnist_metrics_across_clients() to include metrics computed using tff.federated_collect().

This might look something like:

@tff.federated_computation
def aggregate_mnist_metrics_across_clients(metrics):
  return {
      'num_examples': tff.federated_sum(metrics.num_examples),
      'loss': tff.federated_mean(metrics.loss, metrics.num_examples),
      'accuracy': tff.federated_mean(metrics.accuracy, metrics.num_examples),
      'per_client/num_examples': tff.federated_collect(metrics.num_examples),
      'per_client/loss': tff.federated_collect(metrics.loss),
      'per_client/accuracy': tff.federated_collect(metrics.accuracy),
  }

Which will get printed a few cells later when the computation runs:

state, metrics = iterative_process.next(state, federated_train_data)
print('round  1, metrics={}'.format(metrics))

round  1, metrics=<...,per_client/accuracy=[0.14516129, 0.10642202, 0.13972603],per_client/loss=[3.2409852, 3.417463, 2.9516447],per_client/num_examples=[930.0, 1090.0, 730.0]>

Note however: if you want to know the value of a specific client, there is intentionally no way to do that. By design, TFF's language intentionally avoids a notion of client identity; there is desire to avoid making clients addressable.

Zachary Garrett
  • 2,911
  • 15
  • 23
  • If client address is not accessible, how to find and avoid some attacks such as meaningless or incorrect model update? And since client address is inaccessible, I want to analyze the values of different clients, is this reasonable? – ester Jul 05 '19 at 04:28
  • Determining how to defend against "bad" (and even defining what is "bad") updates is an active area of FL research. While TensorFlow Federated avoids a mapping of which client provided which update, its still possible to inspect (and treat differently) the updates on the server (this may change the implementation uses a secure multiple party communication technique though). The implementation of the FederatedAveraging algorithm include in TFF uses tff.federated_mean(), but constructing a custom tff.federated_reduce() that determines whether to include an update is a possible avenue. – Zachary Garrett Jul 05 '19 at 14:39
0

If you want to print something in 'client_update' function you can use tf.print().

Deepali Kushwaha
  • 1
  • 1