How to build a TensorFlow cluster and let each node make a connection to any rest of the nodes (1 to N-1)? I check the code and its implementation is server-client with gRPC. Does that mean I should build a server and a client on each node so that nodes within the cluster can make a connection to each other?
Related:
https://docs.ray.io/en/latest/cluster/key-concepts.html
https://www.tensorflow.org/guide/distributed_training
https://www.tensorflow.org/tutorials/distribute/parameter_server_training
