Stack Overflow
Stack Overflow

Questions tagged [spring-xd]

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Big data applications share many characteristics with Enterprise Integration and Batch applications. Spring has provided proven solutions for building integration and batch applications for more than 6 years now via the Spring Integration and Spring Batch projects. Spring XD builds upon this foundation and provides a lightweight runtime environment that is easily configured and assembled via a simple DSL.

Click here for more information about Spring XD.

565 questions
2
votes
1 answer

Spring XD: How to decide when to use streams or batch jobs?

We currently developing an application that read files from file system/ftp/http and upload this files to mongodb. After that we want to analyze this files and write analyzed documents in mongodb. Requirements for this process: Real-time uploading…
Anton Semahko
  • 21
  • 1
2
votes
1 answer

Poll a JMS Queue from Spring XD source for throttling

I am trying to use XD JMS source to read data from activeMq queue and just log it. My requirement is to read the queue at specific interval only, trying to implement throttling. I need my stream to process only 1 message per second, the queue may…
Ashish C
  • 77
  • 6
2
votes
1 answer

Spring XD: pipe (>) from file source to batch job fails (IllegalArgumentException: Unable to convert provided JSON to Map)

I am trying to pipe the output (file contents) of a Spring XD file source to a simple batch job. This fails with the following exception. It appears that XD is trying to incorrectly use the file contents as JSON job parameters when launching my job.…
Reid Harrison
  • 406
  • 3
  • 9
2
votes
1 answer

Spring XD (Spring Integration DSL based) AWS S3 sink module error

I'm trying to build a Spring XD (Spring Integration DSL based) sink module using the spring-integration-aws extension. My module looks like so: @Configuration @EnableIntegration public class S3Module { @Value("${accessKey:myAccessKey}") …
chadmaughan
  • 578
  • 1
  • 4
  • 12
2
votes
1 answer

Spring XD to read multiple website data

I need to create a website that reads contents of different websites and help to compare them. One of the examples having a similar website http://www.mysmartprice.com/mobile/samsung-galaxy-grand-2-msp3633 This helps us to compare prices of samsung…
Jubin
  • 51
  • 2
2
votes
1 answer

Spring-XD Curator Connection Timeout

In Spring-XD the Curator Connection times out: WARN ConnectionStateManager-0 curator.ConnectionState - Connection attempt unsuccessful after 63021 (greater than max timeout of 60000). Resetting connection and trying again with a new…
Ajit
  • 128
  • 1
  • 8
2
votes
2 answers

How to see all jobs in a cluster using Spring Batch?

I'm trying to determine all the things I need to consider when deploying jobs to a clustered environment. I'm not concerned about parallel processing or other scaling things at the moment; I'm more interested in how I make everything act as if it…
Snekse
  • 15,474
  • 10
  • 62
  • 77
2
votes
3 answers

How to load data from oracle and sql server to HAWQ using Spring XD

Hi I have tables in Oracle and SQL Server. I need to load data from oracle and sql server into Pivotal HAWQ using Spring XD. Couldn't find in documentation.
zniv
  • 166
  • 1
  • 2
  • 12
2
votes
1 answer

Using mappedRequestHeaders in Spring XD

I am trying to simulate the following curl statement curl -k -H "username: Your API Username" -H "password: Your API Password" https://infoconnect1.highwayinfo.govt.nz/ic/jbi/TrafficCameras/REST/FeedService/ How do I use mappedRequestHeaders to…
Teik Hooi Beh
  • 37
  • 1
2
votes
2 answers

Setting http headers processor:http-client

I would like to set User-Agent header when using processor:http-client I see there is mappedRequestHeaders option, but how to use to set User-Agent: Mozilla for example Either I am missing something or spring-xd lacking processor:header-enricher.…
vladsfl
  • 617
  • 8
  • 18
2
votes
0 answers

Spring XD : File sink with date parameter

I have just installed spring xd for doing a POC. My initial goal is to install spring xd on all the application server, collect the logs of the day and dump it to a location in a NFS mount. I have setup spring xd in the distribution mode. My stream…
Gowri Shankar
  • 161
  • 1
  • 3
  • 12
2
votes
1 answer

Spring XD - UDP inside Jobs

I have been using Spring XD for a while for continuous ingestion of sensor data and it works perfectly. The new requirement that I have is the ability to "replay" portions of that data. In my particular case it would be reading from MongoDB (with a…
dmendezg
  • 1,888
  • 1
  • 15
  • 23
2
votes
1 answer

Spring XD -- xd-singlenode start failure

I have set the environment variable spring.profiles.active to "qa" which is the profile in application.yml file which i want to activate. But after setting this environment variable, and starting Sprind XD in xd-singlenode, it throws me the…
VinniK
  • 35
  • 5
2
votes
1 answer

How to run Spring XD on windows platform

What's the best practice for running Spring XD on a Windows platform? Most of what I have seen has been just running it from the command line. This is good for demos but doesn't work well in production environments. Can a Windows Service be create…
zachariahyoung
  • 821
  • 4
  • 17
  • 29
2
votes
0 answers

Spring XD properties-location attribute

I tried to create a stream with this definition: stream create --name test3 --definition "jms --destination=test3 --pubSub=true --subscriptionName=test3 --durableSubscription=true --clientId=cbda83 --outputType=text/plain | filter…
Arun Jose
  • 365
  • 4
  • 16
1 2 3
…
37 38