2

I'm using spring-integration-zeromq and I'm trying to set up with authentication settings.

    @Bean
    ZeroMqChannel zeroMqPubSubChannel(ZContext context, ObjectMapper objectMapper) {
        ZeroMqChannel channel = new ZeroMqChannel(context, true);
        channel.setConnectUrl("tcp://localhost:6001:6002");
        channel.setConsumeDelay(Duration.ofMillis(100));
        channel.setMessageConverter(new GenericMessageConverter());
        channel.setSendSocketConfigurer(socket -> {
            socket.setZAPDomain("global".getBytes());
            socket.setCurveServer(true);
            socket.setCurvePublicKey("my_public_key".getBytes());
            socket.setCurveSecretKey("my_secret_key".getBytes());
        });
        EmbeddedJsonHeadersMessageMapper mapper = new EmbeddedJsonHeadersMessageMapper(objectMapper);
        channel.setMessageMapper(mapper);
        channel.afterPropertiesSet();

        channel.subscribe(m -> System.out.println(m));
        return channel;
    }

However it seems like results of setSendSocketConfigurer are ignored. In the org.springframework.integration.zeromq.channel.ZeroMqChannel sendSocketConnectionConfigurer is inited as an empty lambda and passed as such into prepareSendSocketMono; so me calling setSendSocketConfigurer consequently seem to have no effect as it only replaces a property in ZeroMqChannel instance, but not being applied to an already created by then socket mono. How do I set up authentication properly? Am I missing something?


UPD1

After the fix provided by Artem Bilan, the socket configurers seem to have started being applied to the channel, but communication stopped working. I have applied recommendations and tried out setting up ZeroMqProxy in hope it'll provide me with a workaround, but still no success: even my logging subscription in the same config is not coming through the authentication (though it works if I remove the socket configurers calls)

@Configuration
public class ZeroMQConfig {
    @Bean
    ZeroMqProxy zeroMqProxy(ZContext context, @Value("${zmq.channel.port.frontend}") int frontendPort,
                            @Value("${zmq.channel.port.backend}") int backendPort) {
        ZeroMqProxy proxy = new ZeroMqProxy(context, ZeroMqProxy.Type.SUB_PUB);
        proxy.setExposeCaptureSocket(true);
        proxy.setFrontendPort(frontendPort);
        proxy.setBackendPort(backendPort);
        ZCert cert = new ZCert();
        proxy.setFrontendSocketConfigurer(socket -> {
            socket.setCurvePublicKey(cert.getPublicKey());
            socket.setCurveSecretKey(cert.getSecretKey());
            socket.setCurveServerKey(Z85.decode("my_server_public_key"));
        });
        proxy.setBackendSocketConfigurer(socket -> {
            socket.setCurvePublicKey(cert.getPublicKey());
            socket.setCurveSecretKey(cert.getSecretKey());
            socket.setCurveServerKey(Z85.decode("my_server_public_key"));
        });
        return proxy;
    }

    @Bean
    public ZContext zContext() {
        ZContext context = new ZContext();
        ZAuth auth = new ZAuth(context);
        auth.configureCurve(ZAuth.CURVE_ALLOW_ANY);
        auth.setVerbose(true);
        return context;
    }

    @Bean(name = "zeroMqPubChannel")
    ZeroMqChannel zeroMqPubChannel(ZContext context, ObjectMapper objectMapper, ZeroMqProxy proxy){
        ZeroMqChannel channel = new ZeroMqChannel(context, true);
        channel.setZeroMqProxy(proxy);
        channel.setConsumeDelay(Duration.ofMillis(100));
        channel.setMessageConverter(new GenericMessageConverter());
        EmbeddedJsonHeadersMessageMapper mapper = new EmbeddedJsonHeadersMessageMapper(objectMapper);
        channel.setMessageMapper(mapper);
        return channel;
    }

    @Bean
    @ServiceActivator(inputChannel = "zeroMqPubChannel")
    public MessageHandler subscribe() {
        return message -> System.out.println(message);
    }
}
Tatiana Goretskaya
  • 536
  • 10
  • 25

1 Answers1

1

Yeah... I see you point. This a bug: we must postpone a his.sendSocketConfigurer usage until really an interaction happens with a socket. I'll fix that soon enough.

For now on a couple remarks for your config:

You must not call an afterPropertiesSet() yourself. Let the Spring application context to manage its callbacks for you!

You must not subscribe into the MessageChannel in its bean definition. Instead consider to have a @ServiceActivator(inputChannel = "zeroMqPubSubChannel"). See more info in docs: https://docs.spring.io/spring-integration/reference/html/messaging-endpoints.html#service-activator

Unfortunately there is no way to pass that customization into an internal ZMQ.Socket instance...

UPDATE

The working test with Curve auth in ZeroMQ:

@Test
void testPubSubWithCurve() throws InterruptedException {
    ZContext CONTEXT = new ZContext();
    new ZAuth(CONTEXT).configureCurve(ZAuth.CURVE_ALLOW_ANY).setVerbose(true);

    ZMQ.Curve.KeyPair frontendKeyPair = ZMQ.Curve.generateKeyPair();
    ZMQ.Curve.KeyPair backendKeyPair = ZMQ.Curve.generateKeyPair();

    ZeroMqProxy proxy = new ZeroMqProxy(CONTEXT, ZeroMqProxy.Type.SUB_PUB);
    proxy.setBeanName("subPubCurveProxy");
    proxy.setFrontendSocketConfigurer(socket -> {
        socket.setZAPDomain("global".getBytes());
        socket.setCurveServer(true);
        socket.setCurvePublicKey(frontendKeyPair.publicKey.getBytes());
        socket.setCurveSecretKey(frontendKeyPair.secretKey.getBytes());
    });
    proxy.setBackendSocketConfigurer(socket -> {
        socket.setZAPDomain("global".getBytes());
        socket.setCurveServer(true);
        socket.setCurvePublicKey(backendKeyPair.publicKey.getBytes());
        socket.setCurveSecretKey(backendKeyPair.secretKey.getBytes());
    });
    proxy.afterPropertiesSet();
    proxy.start();

    ZeroMqChannel channel = new ZeroMqChannel(CONTEXT, true);
    channel.setZeroMqProxy(proxy);
    channel.setBeanName("testChannelWithCurve");
    channel.setSendSocketConfigurer(socket -> {
        ZCert clientCert = new ZCert();
        socket.setCurvePublicKey(clientCert.getPublicKey());
        socket.setCurveSecretKey(clientCert.getSecretKey());
        socket.setCurveServerKey(frontendKeyPair.publicKey.getBytes());
    });
    channel.setSubscribeSocketConfigurer(socket -> {
                ZCert clientCert = new ZCert();
                socket.setCurvePublicKey(clientCert.getPublicKey());
                socket.setCurveSecretKey(clientCert.getSecretKey());
                socket.setCurveServerKey(backendKeyPair.publicKey.getBytes());
            }
    );
    channel.setConsumeDelay(Duration.ofMillis(10));
    channel.afterPropertiesSet();

    BlockingQueue<Message<?>> received = new LinkedBlockingQueue<>();

    channel.subscribe(received::offer);
    channel.subscribe(received::offer);

    await().until(() -> proxy.getBackendPort() > 0);

    // Give it some time to connect and subscribe
    Thread.sleep(1000);

    GenericMessage<String> testMessage = new GenericMessage<>("test1");
    assertThat(channel.send(testMessage)).isTrue();

    Message<?> message = received.poll(10, TimeUnit.SECONDS);
    assertThat(message).isNotNull().isEqualTo(testMessage);
    message = received.poll(10, TimeUnit.SECONDS);
    assertThat(message).isNotNull().isEqualTo(testMessage);

    channel.destroy();
    proxy.stop();
    CONTEXT.close();
}
Artem Bilan
  • 113,505
  • 11
  • 91
  • 118
  • Thanks for the prompt reply! Is there a jira/github issue I could use for tracking? – Tatiana Goretskaya Apr 22 '21 at 14:52
  • There is no, since I'm going just to Pull Request it - and it is going to be reviewed and merged pretty quickly. With the proper back-port, of course. Once that PR is opened, I'll post its link here, so you really can track and review. Plus in the end we will ask you to validate the solution on your side against SNAPSHOT we will publish after merging that PR. – Artem Bilan Apr 22 '21 at 14:56
  • 1
    Here is the fix: https://github.com/spring-projects/spring-integration/pull/3553 – Artem Bilan Apr 22 '21 at 16:00
  • So, the versions `5.4.7-SNAPSHOT` & `5.5.0-SNAPSHOT` have been published into the https://repo.spring.io/snapshot. Feel free to try the fix out! – Artem Bilan Apr 22 '21 at 17:00
  • 1
    I was able to verify that authentication is somehow applied now as my messages stopped coming through (it still works without the socket configurers and it works with the same keys in my non-Spring PoC) - not sure if it's because I messed up the auth settings somewhere or if authentication settings are not applied properly. I'll get back if I find something – Tatiana Goretskaya Apr 22 '21 at 18:57
  • Still struggling with authentication. Tried it the other way around - with setting a proxy instead of url (though seems it won't work for me in the long run as it only can set url to localhost?..) - but if I set socket configurers, even my logging subscription stops working. I'll add the new code as UPD1 to the post – Tatiana Goretskaya Apr 28 '21 at 12:04
  • It would be great if you share some simple project to play with. I’d like to learn what is that “auth” in ZeroMQ. You probably have something in raw ZeroMQ API, so I will be able to compare with Spring Integration and cover more with tests. – Artem Bilan Apr 28 '21 at 12:09
  • 1
    https://github.com/aquarellian/springboot_experiments -- main branch conains somewhat working code (no authorization). (Message parsing is broken there because JsonConverter does not allow basic String message, but at least I can see my message in subscribers -- and I have my logic working with my home-baked GenericMessage). https://github.com/aquarellian/springboot_experiments/tree/use_auth contains code that uses authorization via ZCert - no subscriber receives the message. STR: Start SpringBoot app, ZMQServer & ZMQSubscriber, refresh localhost:8080 – Tatiana Goretskaya Apr 28 '21 at 17:53
  • Sorry, that's too complicated to digest. Isn't there a way to make it as simple as possible. Perhaps like just one JUnit test? – Artem Bilan Apr 28 '21 at 19:33
  • Short example would be to only run ZMQPublisher and ZMQSubscriber on the same port with subscriber binding to the port (pass isServer=true to the socket factory) and publisher connecting to it (pass isServer=false). – Tatiana Goretskaya Apr 29 '21 at 16:51
  • Sure! Why don't provide such a project for us then instead? Thank you! – Artem Bilan Apr 29 '21 at 16:52
  • Is this not sufficient? https://github.com/aquarellian/springboot_experiments/blob/master/src/main/java/com/example/process/ZMQSubscriber.java – Tatiana Goretskaya Apr 29 '21 at 16:54
  • No, because I don't know what is that `SocketFactory`. Plus it is hard for me to digest Lombok. But anyway that all would be great if you provide a simple project which I can simply run on my side. Otherwise I'll go and look what is CURVE in ZeroMQ and will try to write my own unit test. But definitely that is going to take some time – Artem Bilan Apr 29 '21 at 16:56
  • SocketFactory is a class right next to ZMQSubscriber and ZMQPublisher in the project: https://github.com/aquarellian/springboot_experiments/blob/master/src/main/java/com/example/process/SocketFactory.java. Lombok used in those classes simply generates constructor for a class setting all the final parameters (@RequiredArgsConstructor) and @SneakyThrows can be abstracted out as runtime exception wrapper over a method that would have to otherwise declare a checked exception. – Tatiana Goretskaya Apr 29 '21 at 17:01
  • I have created a new example with just zmq: https://github.com/aquarellian/zmq. Thank you for your help, I really appreciate it. I wish I could look through spring-integration-zeromq, but I'm relatively new to all 3 zmq, springboot and reactor framework, so altogether it's relatively hard to get a hold of what's happening there. – Tatiana Goretskaya Apr 29 '21 at 17:25
  • 1
    Please, find an UPDATE in my answer. It shows a Curve auth for Proxy as a server and `ZeroMqChannel` as a client. If I mess up with some key, it just doesn't show anything, but apparently it doesn't connect and subscribe... – Artem Bilan Apr 29 '21 at 19:27
  • Thanks a lot! Thanks to you, I've got it working! – Tatiana Goretskaya Apr 30 '21 at 13:25