5

Our server-sided solution makes use of Hazelcast-provided distributed data structures to make available state related to entities that live on particular cluster members.

When a cluster member joins or leaves the cluster, we have a need for the other cluster nodes to be aware of 'what changed': for example, when a cluster member leaves, the other cluster members need to be able to determine what entities have become unavailable as a result of the cluster event, separating that from the changes in availability of entities that have other causes (eg: related to the normal lifecycle behavior of such entities).

A naive implementation could be based on a keeping a record, on each cluster member, that tracks what resources is provided by what cluster member. Such a record could then be used when cluster events are detected. However, this very much feels like re-inventing a wheel.

Can Hazelcast facilitate a solution?

Guus
  • 2,986
  • 2
  • 21
  • 32
  • Arguably naively, I'm thinking that on a low level, Hazelcast data structure implementations have a need for such a mechanism themselves. When data becomes unavailable because of a cluster event, doesn't local memory need to be vacated (eventually), and, conversely, wouldn't the sudden availability of data need to be distributed (in a scenario where a fully replicated data structure is used)? Can that mechanism be leveraged? – Guus Nov 12 '20 at 13:26
  • Not sure what you meant by data becoming unavailable and the need for local memory to be vacated, for when a member leaves the cluster due to network disruptions and wants to come back later, what happens to its data is defined by the merge policy. Default is - wipe everything before joining. As for the remaining nodes, data from lost node is recovered from backups on remaining nodes and this happens automatically, as soon as node leaving is confirmed. – wildnez Dec 22 '20 at 22:04
  • Let me elaborate on what I mean with 'becoming unavailable': Each of our cluster nodes add resources to the cluster as a whole that are specific to that node. As a simplified example: a user session, representing a uses connected via a TCP connection to a particular node. We're keeping the session representation in a cache that's shared across the cluster. When a node is disconnected from the cluster, we have a need to inform entities on other cluster nodes what sessions (provided by the disconnected node) are now no longer available. Can we utilize Hazelcast for this? – Guus Jan 04 '21 at 16:35
  • One way I see you can achieve this is by using MultiMap where a key would be a member ID and value would be all the sessions that were generated on that node. So basically when a user session is created, also store that in the MultiMap against the member/node ID. Then when a node leaves, you have MembershipListener on remaining nodes which can extract all sessions from the MultiMap against the node that has just left the cluster. Makes sense? – wildnez Jan 05 '21 at 00:35
  • You can also look into using web session replication of Hazelcast, here is the documentation: https://docs.hazelcast.org/docs/4.1.1/manual/html-single/index.html#web-session-replications – wildnez Jan 05 '21 at 00:38

1 Answers1

0

Such granular level housekeeping is neither feasible nor recommended for any distributed system because it adds significant overhead. In case of Hazelcast, there are provisions that can give you some insight into the state of the data when cluster topology changes. Try PartitionLostListener and/or Migration Listener.

wildnez
  • 1,078
  • 1
  • 6
  • 10
  • Unless I'm misunderstanding, your response seems to relate to data that is generically kept in a data grid. The context of my question is different, I think. In our use-case, each cluster member maintains a set of resources that is specific to that cluster member: it can't be transferred over the cluster. We keep a representation of the resource in a Hazelcast distributed data structure. Individual resources can, but need not be, interested in the availability of another. What we're looking for is a trigger to be able to send an event that indicates a change in availability of resources. – Guus Nov 12 '20 at 13:19