Stack Overflow
Stack Overflow

Questions tagged [spring-xd]

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Big data applications share many characteristics with Enterprise Integration and Batch applications. Spring has provided proven solutions for building integration and batch applications for more than 6 years now via the Spring Integration and Spring Batch projects. Spring XD builds upon this foundation and provides a lightweight runtime environment that is easily configured and assembled via a simple DSL.

Click here for more information about Spring XD.

565 questions
2
votes
1 answer

Is it possible to JOIN multiple streams with Spring XD?

I need to define multiple streams from multiple sources and then join them into one. After reading the whole documentation of Spring XD, I did not find any predefined module that can do a join (based on sliding windows for example). Is there any…
user3906228
  • 21
  • 1
2
votes
1 answer

spring-xd how to create stream with multi processor

I have a question is how to create stream with multi processor. For ex: stream create --name multiModuleTest --definition "tcp | processor1 | processor2 |file" --deploy Or stream create --name multiModuleTest --definition "(tcp, http) | processor1…
Tu Pham
  • 41
  • 1
  • 4
2
votes
1 answer

Spring-xd redis message bus removing headers from the message

I am using redis as my message store for spring-xd distributed env. I am setting the message header as below, oufinalTreeMapsg1 = MessageBuilder.withPayload(treeMap).setHeader("columnNames", columnNames).build(); while checking the message received…
Madhu Mohan Kommu
  • 335
  • 2
  • 13
2
votes
4 answers

spring-xd how to use source:file read line one by one

I have a stream that watch output of multi file in a directory, process data and put it to HDFS. Here is my stream creat command: stream create --name fileHdfs --definition "file --dir=/var/log/supervisor/ --pattern=tracker.out-*.log…
Tu Pham
  • 41
  • 1
  • 4
2
votes
1 answer

spring XD jobs and Cassandra

I'm new to spring XD ... I'm trying to create a job that will read a csv file and use spring-data-cassandra to insert data in a Cassandra database. Everything is fine in my batch project. I can package my batch and deploy it to spring xd into…
Franck MARCHAND
  • 41
  • 5
2
votes
1 answer

spring xd source moodule does not processing files parallelly

I am working on Spring XD M7 version. I have created a steam source | processor | sink. I have total 5 files in my source folder.When my stream deployed, it is processing sequentially where I have 3 nodes spring xd cluster with redis configured. My…
Madhu Mohan Kommu
  • 335
  • 2
  • 13
2
votes
1 answer

Any concept of a global variable in streams in spring xd?

Scenario: A stream definition in spring xd has the following structure: jms | filter | transform | hdfs In the filter module, I fire a query to a database to verify if the current message is applicable for further processing. When the condition is…
Arun Jose
  • 365
  • 4
  • 16
2
votes
1 answer

How to configure Redis in Spring XD distributed runtime?

The Spring XD documentation (http://docs.spring.io/spring-xd/docs/1.0.0.RC1/reference/html/) recommends Zookeeper to be run in ensemble so that Zookeeper is highly available. There is not lot of details about Redis about high availability. If I…
SathiyaS
  • 246
  • 3
  • 11
2
votes
1 answer

Is it possible for Spring-XD to listen to more than one JMS broker at a time?

I've managed to get Spring Xd working for a scenario where I have data coming in from one JMS broker. I potentially am facing a scenario where data ingestion could happen from different sources thereby needing me to connect to different…
Arun Jose
  • 365
  • 4
  • 16
2
votes
1 answer

How can one create a Spring-XD processor module to filter messages to the output channel

There are lots of examples of how to write transformers etc in java but nothing about filters (except the script type filters, but I want to use a java method). I'd like to create a custom java filter to filter the payload of a message from a source…
Ron Tuffin
  • 53,859
  • 24
  • 66
  • 78
2
votes
1 answer

Stream | Processors and thread model

I was wondering how does spring-xd deals with processors in a stream. What I really would like to know is if processors are blocking code, or they are related to how reactor (https://github.com/reactor/reactor/wiki/Processor) deals with…
Vinicius Carvalho
  • 3,994
  • 4
  • 23
  • 29
2
votes
2 answers

How does one run Spring XD in distributed mode?

I'm looking to start Spring XD in distributed mode (more specifically deploying it with BOSH). How does the admin component communicate to the module container? If it's via TCP/HTTP, surely I'll have to tell the admin component where all the…
DeejUK
  • 12,891
  • 19
  • 89
  • 169
1
vote
0 answers

I am trying to run a Groovy script from Spring XD shell. But script does not trigger

I am trying to extract data from SQL SERVER into HDFS using groovy script. The stream definition deploys the stream bu the script is not getting fetched after deployment. Stream definition : stream create --name WhatIFScenarioRun --definition…
Pritam Mohanty
  • 11
  • 3
1
vote
0 answers

How to make the job restartable in Spring xD

The jobs under Executions section of Jobs in Spring xd admin ui has disable restart button. How will this get enabled? I tried making the job itself restartable by supplying the restartable="true"
abhineet
  • 195
  • 1
  • 1
  • 11
1
vote
2 answers

Spring-xd Redis sink with Collection type MAP

Im trying to implement Redis sink with specific to collection type = MAP stream create tTEST_GF_SINK --definition "trigger --initialDelay=0 --fixedDelay=1 --timeUnit=MINUTES --payload= 'new Date().toString()' --outputType=application/json | …
keerthi
  • 58
  • 8
1 2 3
…
37 38