Stack Overflow
Stack Overflow

Questions tagged [spring-xd]

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Big data applications share many characteristics with Enterprise Integration and Batch applications. Spring has provided proven solutions for building integration and batch applications for more than 6 years now via the Spring Integration and Spring Batch projects. Spring XD builds upon this foundation and provides a lightweight runtime environment that is easily configured and assembled via a simple DSL.

Click here for more information about Spring XD.

565 questions
2
votes
2 answers

How to route messages from kafka-sink to multiple topics

I have a spring-xd http-processor module with http-outbound-gateway that has a errorChannel and outputChannel. Any message with HTTP 200 comes to outputChannel, and rest of them land in failureChannel. Right now, the http-processor module connects…
Vidhya
  • 57
  • 8
2
votes
0 answers

Spring XD Batch Job remote partitioning

I'm new to Spring XD so excuse my ignorance. I have a spring batch job that runs fine as standalone. I need to port it to spring XD, but with remote partitioning. I have 3 XD containers (and 1 admin). I have a redis server and a ZK. All 3…
olumide adebayo
  • 68
  • 5
2
votes
1 answer

Spring-xd container(1.3.1) compatibility with Spark 1.6.1+ Hadoop 2.7.2

I am currently working on a spring-xd runtime container, I use: Spark 1.6.1+ Hadoop 2.7.2 + Spring-xd 1.3.1 My questions: Are these versions compatible to spring-xd container (1.3.1)? Can these be deployed as a spring-boot in a clustered…
Praveen Kumar K S
  • 3,024
  • 1
  • 24
  • 31
2
votes
0 answers

Spring XD is dying?

Based on github graphics looks like Spring XD is slowly dying. Is Spring Cloud Data Flow going to replace Spring XD in a near future?
Serge
  • 2,574
  • 18
  • 26
2
votes
1 answer

Spring Cloud DataFlow documentation has missing sections

I am reading right now the Spring Cloud DataFlow documentation, but it turn out that the sections contained in PART IV (Modules) has some kind of error, because the whole section is fill up with this kind of text Unresolved directive in…
Jose Alejandro Niño Mora
  • 43
  • 3
2
votes
1 answer

Spring XD OAuth Sink unit test Unable to obtain a new access token for resource 'null'

I'm trying to write an integration test for a sink I'm creating for Spring XD. Here is my ModuleConfiguration.java @Configuration @EnableIntegration public class ModuleConfiguration { private OAuth2AccessToken token; @Bean public…
gary69
  • 3,620
  • 6
  • 36
  • 50
2
votes
0 answers

Not all messages are received for a Spring Aggregator

I have an aggregator that is supposed to take all messages with a certain correlation key (in the example below I hardcoded this to 10, because the corresponding Splitter generates 10 messages). I can see many messages via logging, and all 10…
Edmund Bayerle
  • 55
  • 3
2
votes
1 answer

How to configure a Spring InboundChannelAdapter for Spring XD?

I have a custom Spring XD source that I want to use: package com.my.springproject; import org.springframework.integration.annotation.InboundChannelAdapter; import org.springframework.integration.annotation.Poller; public class…
Edmund Bayerle
  • 55
  • 3
2
votes
1 answer

How to use Spring XD Module Options in SpEL expression of Spring Integration DSL?

I'm implementing a custom Spring XD module (processor). I've added Module Options to it, they can be easily referenced in ModuleConfiguration via @Value("${varName}") syntax. However, I can't use them directly in SpEL expressions of the flow…
Max Romanovsky
  • 2,824
  • 5
  • 30
  • 37
2
votes
1 answer

How to implement a Spring XD sink?

So far I have implemented Spring XD processors, e.g. like this: @MessageEndpoint public class MyTransformer { @Transformer( inputChannel = "input", outputChannel = "output" ) public String transform( String payload ) { ... …
Arne
  • 2,624
  • 3
  • 24
  • 45
2
votes
1 answer

How to configure Spring XD JMS source to use DefaultMessageListenerContainer?

To make the JMS topic subscription durable, it seems I need to make sure DefaultMessageListenerContainer (instead of the default SimpleMessageListenerContainer) is used stream definition contains "durableSubscription=true acknowledge=transacted…
nanaboo
  • 365
  • 4
  • 14
2
votes
1 answer

Spring XD - Change File Source directory at runtime

I would like to change configuration of a File Source at runtime. From what I see in docs and in source definition, directory is statically defined (i.e. no expressions), see the bean definition. I know that using plain Spring Integration it's…
execc
  • 1,083
  • 12
  • 25
2
votes
1 answer

encoding password in spring-integration or xd

I want to encode my acccessKey and secretKey in options during stream definition is there any utility for the same in xd or integration?I want to encode the key in options
constantlearner
  • 5,157
  • 7
  • 42
  • 64
2
votes
1 answer

How to identify unprocessed messages after KAFKA topic consumption

Scenario : Stream create [StreamName] --definition " Kafka -zkconnect=10.10.10.1:2181 --topic= | MyCompositeModule " --deploy We are running this stream in distributed mode and redis is the transport bus. Per my understanding, kafka…
keerthi
  • 58
  • 8
2
votes
0 answers

How can I define a Java system property in a Spring XD stream?

For some ad hoc testing of a custom Spring XD processor, I would like to specify a Java system property during the definition of the stream to be passed to the processor. Is this possible?
Arne
  • 2,624
  • 3
  • 24
  • 45
1 2
3
…
37 38