How do I convert this spring-integration configuration from XML to Java?

Question

This particular piece makes sense to implement in the application rather than XML because it is a constant across the entire cluster, not localized to a single job.

From dissecting the XSD, it looks to me like the xml for int-kafka:outbound-channel-adapter constructs a KafkaProducerMessageHandler.

There is no visible way to set the channel, the topic, or most of the other attributes.

Note to potential downvoters - (rant on) I have been RTFM'ing for a week and am more confused than when I started. My choice of language has graduated from adjectives through adverbs, and I'm starting to borrow words from other languages. The answer may be in there. But if it is, it is not locatable by mere mortals. (rant off)

XML configuration:

<int-kafka:outbound-channel-adapter id="kafkaOutboundChannelAdapter"
                                    kafka-template="kafkaTemplate"
                                    auto-startup="false"
                                    channel="outbound-staging"
                                    topic="foo"
                                    sync="false"
                                    message-key-expression="'bar'"
                                    send-failure-channel="failures"
                                    send-success-channel="successes"
                                    partition-id-expression="2">
</int-kafka:outbound-channel-adapter>

If so, then I would expect the java config to look something like this:

@Bean
public KafkaProducerMessageHandler kafkaOutboundChannelAdapter () {
    KafkaProducerMessageHandler result = new KafkaProducerMessageHandler(kafkaTemplate());

    result.set????? ();    // WTH?? No methods for most of the attributes?!!!

    return result;
}

EDIT: Additional information about the high level problem being solved

As a part of a larger project, I am trying to implement the textbook example from https://docs.spring.io/spring-batch/4.0.x/reference/html/spring-batch-integration.html#remote-partitioning , with Kafka backing instead of JMS backing.

I believe the final integration flow should be something like this:

partitionHandler -> messagingTemplate -> outbound-requests (DirectChannel) -> outbound-staging (KafkaProducerMessageHandler) -> kafka

kafka -> executionContainer (KafkaMessageListenerContainer) -> inboundKafkaRequests (KafkaMessageDrivenChannelAdapter) -> inbound-requests (DirectChannel) -> serviceActivator (StepExecutionRequestHandler)

serviceActivator (StepExecutionRequestHandler) -> reply-staging (KafkaProducerMessageHandler) -> kafka

kafka -> replyContainer (KafkaMessageListenerContainer) -> inboundKafkaReplies (KafkaMessageDrivenChannelAdapter) -> inbound-replies (DirectChannel) -> partitionhandler

score 2 · Accepted Answer · answered Mar 27 '19 at 20:04

2

Not sure what you mean that they are missed, but this is what I see in the source code of that KafkaProducerMessageHandler:

public void setTopicExpression(Expression topicExpression) {
    this.topicExpression = topicExpression;
}

public void setMessageKeyExpression(Expression messageKeyExpression) {
    this.messageKeyExpression = messageKeyExpression;
}

public void setPartitionIdExpression(Expression partitionIdExpression) {
    this.partitionIdExpression = partitionIdExpression;
}

/**
 * Specify a SpEL expression to evaluate a timestamp that will be added in the Kafka record.
 * The resulting value should be a {@link Long} type representing epoch time in milliseconds.
 * @param timestampExpression the {@link Expression} for timestamp to wait for result
 * fo send operation.
 * @since 2.3
 */
public void setTimestampExpression(Expression timestampExpression) {
    this.timestampExpression = timestampExpression;
}

and so on.

You also have access to the super class setters, for example a setSync() for your XML variant.

The input-channel is not a MessageHandler responsibility. It goes to the Endpoint and can be confgigured via @ServiceActivator alongside with that @Bean.

See more info in the Core Spring Integration Reference Manual: https://docs.spring.io/spring-integration/reference/html/#annotations_on_beans

Also there is very important chapter in the beginning: https://docs.spring.io/spring-integration/reference/html/#programming-tips

In addition it might be better to consider to use Java DSL instead of direct MessageHandler usage:

             Kafka
                .outboundChannelAdapter(producerFactory)
                .sync(true)
                .messageKey(m -> m
                        .getHeaders()
                        .get(IntegrationMessageHeaderAccessor.SEQUENCE_NUMBER))
                .headerMapper(mapper())
                .partitionId(m -> 0)
                .topicExpression("headers[kafka_topic] ?: '" + topic + "'")
                .configureKafkaTemplate(t -> t.id("kafkaTemplate:" + topic))
                .get();

See more info about Java DSL in the mentioned Spring Integration Docs: https://docs.spring.io/spring-integration/reference/html/#java-dsl

answered Mar 27 '19 at 20:04

Artem Bilan

113,505
11
91
118

Thank you Artem. I really do appreciate your help. I will review these documents. I just want to wire up a "simple" connection to kafka, and I'm having to learn two new programming languages that are embedded within java, with unstated assumptions about prior knowledge. In the end I expect a couple of dozen lines of code, but it's very frustrating getting to that point. – pojo-guy Mar 27 '19 at 20:20
Well, for simple use-case you may consider just to use `KafkaTemplate` or `KafkaProducer` directly. However in the end you'll find that Spring Integration is really useful tool to build complex messaging integration solutions. – Artem Bilan Mar 27 '19 at 20:24
I agree, it looks like a good (and necessary) tool, or I wouldn't have stuck it out this far. Unfortunately, the documents read like a feature spec rather than a how-to, so unless you are intimately familiar with the internals and plan to spend a significant portion of your career specializing in it, you have no way of knowing how to wire things up or even find out how to wire them up. I started with DSL examples and backed down to something I could understand over time. The DSL may look great to someone familiar with it, but I find it completely unreadable. – pojo-guy Mar 27 '19 at 20:34
I might add that the "cafe" sample code in the DSL document is exactly the sort of thing that comes out as a feature list. Starting out by saying "The DSL provides a simple way to embed Spring Integration Message Flows" , then launching into the cafe example which is anything but simple, really does not do service people new to the product or the DSL. – pojo-guy Mar 27 '19 at 20:51
Regarding this DSL example: if I read it correctly, everything up to the "get" is building a specification for the bean that is to be produced. The "get" at the end is where the configuration is executed and the bean is returned. Is this correct? – pojo-guy Mar 28 '19 at 13:51
1

The `get()` returns an instance of an object which will become a bean in Spring Container. There are more automatic work is done afterwards by Spring. – Artem Bilan Mar 28 '19 at 14:07
I did end up using the DSL form of configuration for this bean. Thanks to your last clarification the DSL is becoming understandable to me now. Is there an easier way to learn than than trial and error? I've been working for a week on implementing a textbook example (https://docs.spring.io/spring-batch/4.0.x/reference/html/spring-batch-integration.html#remote-partitioning), using Kafka instead of JMS. I suspect that I am missing some core concept to make this so difficult. Thank you – pojo-guy Mar 28 '19 at 16:49
1

Not sure how to help from here, but you indeed need to make yourself familiar with all the EIP (https://www.enterpriseintegrationpatterns.com/) first of all to have a bag of principles and concepts, then come to our Java DSL chapter to see all those relevances: https://docs.spring.io/spring-integration/reference/html/#java-dsl. The `Using Protocol Adapters` paragraph should shed some light what is `namespace factory` and how to use it. – Artem Bilan Mar 28 '19 at 16:59

How do I convert this spring-integration configuration from XML to Java?

1 Answers1

Linked