Stack Overflow
Stack Overflow

Questions tagged [spring-xd]

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Big data applications share many characteristics with Enterprise Integration and Batch applications. Spring has provided proven solutions for building integration and batch applications for more than 6 years now via the Spring Integration and Spring Batch projects. Spring XD builds upon this foundation and provides a lightweight runtime environment that is easily configured and assembled via a simple DSL.

Click here for more information about Spring XD.

565 questions
0
votes
3 answers

Spring XD Distributed Environment

I am working on Spring XD and GemFire XD. I want to understand how Spring XD's distributed environment works. I know spring xd uses either redis or rabittmq as the transport. I am clear about this, I have install spring xd and rabittmq on one…
Madhu Mohan Kommu
  • 335
  • 2
  • 13
0
votes
1 answer

SpringXD stream with Tail source reads ALL the lines in file again and again instead of just last n lines

stream create --name TailTest --definition "tail --name=/Users/name/Documents/SpringXD/PoC/input/Try.txt --outputType=text/plain --lines=1 |file --name=output --dir=/Users/name/Documents/SpringXD/PoC/output --mode=APPEND" --deploy Here I see that…
user3754631
  • 3
  • 1
0
votes
0 answers

Is it possible to configure pool size for batch job workflows database in spring-xd

I have configured Sql Server to be used as batch job workflows database. I followed the suggestion in documentation and it works well. I wanted to know if the datasource connection is pooled so I can connect to XD in debug mode and catch one of the…
jbrunodomingues
  • 33
  • 4
0
votes
0 answers

GET batch job execution results from Spring XD

Spring XD has a very interesting API for providing batch jobs creation/deployment/undeployment. But, I was wondering why Spring XD is not proposing in its REST Api a GET method to access the result dataset of a job execution (and step execution)…
rico
  • 1,843
  • 2
  • 24
  • 41
0
votes
1 answer

spring-xd running via eclipse documentation

I have been able to start the spring-xd single node by running the SingleNodeApplication via eclipse however when I try to browser to admin application it gives 403 error. Before I go ahead and spend time investigating the cause of it I would like…
dallaybatta
  • 1
  • 3
0
votes
1 answer

Spring XD with Groovy - Real time Analysis

We are using Spring XD, getting some JSON data , need to create tap and need to do sum of the records by using groovy script.. also need to some aggregate function like Min, Max by using groovy script.. Any one having idea like how to do this..
user3656330
  • 171
  • 1
  • 4
  • 9
0
votes
3 answers

Loading data to Hadoop 2.2 with Spring-XD

I'm trying to create stream of data from multiple sources and load it into hdfs. For proof of concept i'm using single node hadoop instalation and Spring-XD on a same machine. Hadoop was deployed with Ambari and reports to be functional and healthy…
JagdCrab
  • 635
  • 2
  • 9
  • 22
0
votes
1 answer

How does Spring-XD handle job execution

I can't get the information out of the documentation. Can anyone tell me how Spring-XD executes jobs? Does it assign a job to a certain container and is this job only executed on the container it is deployed to, or is each job execution assigned to…
Peter Rietzler
  • 471
  • 5
  • 11
0
votes
1 answer

spring xd stream definition dynamic parameters

We need to use the HDFS sink to store some data in HDFS. When creating the stream definition, I can use the "directory" property to specify the hdfs directory where we want to save the file. However, in our use case we want the directory to be …
Rodrigo
  • 1
0
votes
2 answers

spring XD rabbit source module fails to process messages, first message stays unacknowledged

I am trying simple spring XD application to load log events in HDFS. I have configured the target application with the spring-ampq/rabbit log4j appender (the org.springframework.amqp.rabbit.log4j.AmqpAppender Class) to pump log messages to a…
Suhas Gogate
  • 1
0
votes
1 answer

Spring XD stream creates only empty .tmp files

I'm trying to get Spring-XD working with Hortonworks Sandbox VM. Everything went smooth until first, test stream: xd:>stream create --name ticktockhdfs --definition "Time | HDFS" xd:>stream destroy --name ticktockhdfs xd:>hadoop fs ls…
korro
  • 499
  • 9
  • 17
0
votes
1 answer

Why is my spring.datasource configuration not being picked up as expected

I have a batch job which runs perfectly well in standalone mode. I converted the same to a spring xd batch job. I am using spring xd version 1.0.0.M5. Some issues I face: (i) I do not want to use hsqldb as my spring.datasource. I wanted to switch…
user3304825
  • 107
  • 2
  • 12
0
votes
2 answers

Spring XD or Integration to parse log stats in real time

I have a dashboard web application that currently starts a thread and tails a log file, even time a line is added to the file the tailer picks it up, parses it and then publishes an event around the application, which in turn does things like send…
Zac Tolley
  • 2,340
  • 4
  • 19
  • 22
0
votes
1 answer

Spring XD integration with hortonworks sandbox

I am trying to do the Hortonworks tutorial "Using Spring XD to stream Tweets to Hadoop for Sentiment Analysis" As part of the step I am using the command: xd>xd-shell --hadoopDistro hadoop20 xd> hadoop config fs --namenode…
user3049935
  • 11
  • 1
0
votes
1 answer

Spring-XD: Deployment of modules to certain containers

Three questions regarding deployment of modules to Spring XD container: For certain sources and sinks it's necessary to say to which container a module should be deployed. Let's say we have a lot of containers on different machines, and we want to…
Tobias Flohre
  • 300
  • 3
  • 9
1 2 3
…
37
38