0

I've been learning Spring Integration for about a month now, and it suddenly dawned on me: what if you have the requirement to only have one instance of an IntegrationFlow at any point in time, but you have multiple instances of the Spring application running in the same environment?

Meaning, I might have, say, 3 nodes/instances of my application running in the dev environment, and all 3 have @Configuration source files defining the same IntegrationFlow bean. But I only want one of the 3 to be running the flow at a time. This sounds like a problem solved in the leader election space. I'm tempted to use something like Consul or even ZK but figured I would reach out to the community in case SI or even Spring in general can offer an easier solution that doesn't come with all the operational overhead and infrastructure of running those consensus systems.

Thanks in advance for any-and-all steering!

hotmeatballsoup
  • 385
  • 6
  • 58
  • 136

1 Answers1

1

See the documentation:

https://docs.spring.io/spring-integration/docs/current/reference/html/endpoint.html#endpoint-roles

Starting with version 4.2, endpoints can be assigned to roles. Roles let endpoints be started and stopped as a group. This is particularly useful when using leadership election, where a set of endpoints can be started or stopped when leadership is granted or revoked, respectively. For this purpose the framework registers a SmartLifecycleRoleController bean in the application context with the name IntegrationContextUtils.INTEGRATION_LIFECYCLE_ROLE_CONTROLLER.

and

https://docs.spring.io/spring-integration/docs/current/reference/html/endpoint.html#leadership-event-handling

Groups of endpoints can be started and stopped based on leadership being granted or revoked, respectively. This is useful in clustered scenarios where shared resources must be consumed by only a single instance. An example of this is a file inbound channel adapter that is polling a shared directory. ...

Currently, Zookeeper and Hazelcast are supported.

https://docs.spring.io/spring-integration/docs/current/reference/html/zookeeper.html#zk-leadership

https://docs.spring.io/spring-integration/docs/current/reference/html/hazelcast.html#hazelcast-leader-election

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • So we have a role-based approach and a distributed lock (leader)-based approach. And it looks like the latter (lock) has a default Spring implementation as well as other ones supported by ZK and Hazelcast. I **know** the lock-based approach will work, but I'm trying to understand how the role-based approach would. How could I leverage it so that in a cluster of, say, 5 JVMs/servers, only 1 of them is configured to have the right role that grants "access" to the `IntegrationFlow`, but the other 4 don't? – hotmeatballsoup Jan 06 '23 at 01:21
  • 1
    I think you are missing the fact that `IntegrationFlow` has no value at runtime, and more important to pay attention to endpoints. See `ConsumerEndpointSpec.role()` to meet those leader election requirements. You just provide a leader initiator impl in your project according to available environment: ZK, HZ or any other supporting distributed locks. The `SmartLifecycleRoleController` is there in the application context anyway. So, whenever leader event happens, the respective endpoints in the role are going to be stopped or started. – Artem Bilan Jan 06 '23 at 15:08
  • Hmmm, I sincerely appreciate both your answers + comments here (both of you!) but I'm still not seeing the forest through the trees and connecting the dots, even after looking at these links. The problem is that the examples in the docs aren't SSCCEs, so they are just bits and pieces of an example. Is anyone aware of an open source example (even a GH gist) that leverages these role + leader election features as a fully-working example? – hotmeatballsoup Jan 06 '23 at 15:32
  • 2
    Let's see if this one helps you somehow: https://github.com/artembilan/sandbox/tree/master/leader-election! If it is OK, we can promote it into official samples repo: https://github.com/spring-projects/spring-integration-samples/ – Artem Bilan Jan 09 '23 at 15:24