2

In Erlang, a cluster of connected machines has some soft limitations on how many machines you can have without suffering from too many messages being sent for the gossip protocol to work properly.

This question has similar statements. I have also seen some benchmarks of Riak running ok with 200 nodes (can't find the link but I think it was a Basho test).

I understand that a 200 Erlang node cluster is capable of REALLY heavy loads. I am just wondering on alternatives if that limit becomes a bottleneck.

It seems the answer to a bigger scale than that (which would be huge) is to isolate your system in smaller isolated clusters, much like a micro services approach.

My question is: which is the recommended way of making this communication?

I don't want to come up with a REST service because of the unnecessary weight of marshling/unmarshiling JSON, HTTPS handshakes and other protocol bound operations (load balancers and etc).

BERT-RPC seemed like the best fit but its development seems stale. NkCluster seems to have implemented something similar but is there an Erlang/Elixir/Joxa/LFE way of doing this?

Community
  • 1
  • 1
Olinasc
  • 1,261
  • 1
  • 12
  • 20
  • 5
    It's an interesting question, but I suspect there is no "standard practice" because any problem operating at that scale would be so unique as to require it's own special engineering. – Fred the Magic Wonder Dog Oct 01 '15 at 17:24

1 Answers1

0

What you may be looking for is to set nodes as hidden (http://www.erlang.org/doc/reference_manual/distributed.html#id87289) if you are trying to have a large number of connected nodes that can communicate to each other through explicit connections, but don't want them to be seen otherwise due to propagation.

This allows nodes to connect, but not be spread about through the gossip protocol, so the nodeA knows about the connected hidden node nodeB, but nodeA doesn't tell nodeC about about nodeB when they exchange information about the network topology.

Proctor
  • 61
  • 3
  • That way you are proposing to have a single large cluster right? Intra cluster would not help with hidden nodes or I am not seeing how to plug them together in a highly available and scalable way. – Olinasc Oct 07 '15 at 17:15