Can't set hazelcast ManagedContext when using Jet.bootstrappedInstance()

Question

I want to run hazelcast-jet embedded in my spring application (ie cluster is in process / NOT remote). I also want my jet jobs to be able to access spring beans from the ApplicationContext (see jet/spring docs). I don't want to have to configure the classpath on each jet job, I want to use the JVM's bootstrap classpath.

I think I need to use Jet.bootstrappedInstance() to get a JetInstance but this does NOT allow me to set the ManagedContext on the underlying HazelcastInstance. The docs from Jet.bootstrappedInstance() detail the way in which I want to interact with Jet

When you submit a job to a Jet instance that runs locally in your JVM, it will have all the dependency classes available

I've tried this:

@Bean
public ManagedContext managedContext() {
   return new SpringManagedContext();
}
@Bean
public JetInstance jet(ManagetContext managedContext, List<Pipeline> pipelines) {   
   JetInstance jet = Jet.bootstrappedInstance();
   jet.getHazelcastInstance().getConfig().setManagedContext(managedContext);
   ...
   return jet;
}

But I get the following exception

Caused by: java.lang.UnsupportedOperationException: Unsupported operation
    at com.hazelcast.internal.dynamicconfig.DynamicConfigurationAwareConfig.setManagedContext(DynamicConfigurationAwareConfig.java:981)

Please note that the normal way of calling setManagedContext(...) is like this

   JetConfig config = new JetConfig();
   config.configureHazelcast(hz -> hz.setManagedContext(managedContext));
   JetInstance jet = Jet.newJetInstance(config);

I can't find a way of combining Jet.bootstrappedInstance() with Config.setManagedContext(ManagedContext)

As a workaround, I've copied code from JetBootstrap.createStandaloneInstance() (a private method) and added hzConfig.setManagedContext(managedContext)

  JetConfig config = ConfigProvider.locateAndGetJetConfig();
  Config hzConfig = config.getHazelcastConfig();
  
  // turn off all discovery to make sure node doesn't join any existing cluster
  hzConfig.setProperty("hazelcast.wait.seconds.before.join", "0");
  hzConfig.getAdvancedNetworkConfig().setEnabled(false);
  hzConfig.setManagedContext(managedContext);
  
  JoinConfig join = hzConfig.getNetworkConfig().getJoin();
  join.getAutoDetectionConfig().setEnabled(false);
  join.getMulticastConfig().setEnabled(false);
  join.getTcpIpConfig().setEnabled(false);
  join.getAwsConfig().setEnabled(false);
  join.getGcpConfig().setEnabled(false);
  join.getAzureConfig().setEnabled(false);
  join.getKubernetesConfig().setEnabled(false);
  join.getEurekaConfig().setEnabled(false);
  join.setDiscoveryConfig(new DiscoveryConfig());
  JetInstance jet = Jet.newJetInstance(config);

score 3 · Answer 1 · answered Oct 21 '21 at 09:11

3

I want to run hazelcast-jet embedded in my spring application (I don't want to submit jobs to a remote jet cluster)

In this case using bootstrapInstance() doesn't make sense. It was designed to work with the bin/jet submit command where it creates a client instance according to the config/hazelcast-client.yaml or standalone instance for testing from your IDE.

For using Jet embedded you can use the annotation based configuration or alternatively you can use Hazelcast Jet Spring Boot Starter.

answered Oct 21 '21 at 09:11

František Hartman

14,436
2
40
60

I am using spring annotations to configure my beans (see code above). If I use the `JetInstance` logic from your link, each time I submit a job I have to make sure it has the correct classes (via `JobConfig.addClass(...)` or `JobConfig.addPackage(...)` etc). I want to use the `JetBootstrap.createStandaloneInstance()` logic (see my workaround) which means I don't need to configure the classpath of each job, instead the job has access to the bootstrap classpath – lance-java Oct 21 '21 at 10:08
So do you submit the job from within the application or from outside? – František Hartman Oct 21 '21 at 10:32
I do everything inside, I'm basically trying to use jet in a similar way to `apache-camel` or `spring-integration` – lance-java Oct 21 '21 at 10:50
If you submit the job from within the same JVM you don't need to add any classes or packages, they are simply on classpath. – František Hartman Oct 21 '21 at 11:04
Incorrect, I get ClassNotFoundExceptions – lance-java Oct 21 '21 at 11:37
In that case could you please file a bug with a reproducer at https://github.com/hazelcast/hazelcast/issues, or share the full reproducer here, that's also fine. – František Hartman Oct 21 '21 at 11:42
I can't seem to reproduce which is strange... unfortunately I didn't commit when I got the exception. All seems to be working now. My attempt at a reproduction here https://gist.github.com/uklance/e8b88da8028a7e391da616769d5745b4 – lance-java Oct 22 '21 at 21:39

Can't set hazelcast ManagedContext when using Jet.bootstrappedInstance()

1 Answers1