Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
0 answers

Step Failure not reported by Composed Task Runner or reflected in Spring Cloud Dataflow Tables

Currently we are using Spring Cloud Dataflow to run a sequence of apps we have created based on a definition. Each of the apps we have made are spring batch jobs, with individual steps. The current issue we are having is that when one of these steps…
Silads
  • 7
  • 5
0
votes
1 answer

Spring Cloud dataflow: Register new custom kryo serializer

I am creating a system using cloud-dataflow, I have a source and a transformer. We like to use kryo, but some of our classes require custom kryo serializers, I have written serializers before. We are now using spring-integration-core-4.3.11, and…
ka2
  • 151
  • 2
  • 12
0
votes
2 answers

Tables for JdbcTaskExecutionDao

I'm using @EnableTask with JdbcTaskExecutionDao and I can't find SQLs to create tables in my database (SQL Server). Where I can find them?
MAGx2
  • 3,149
  • 7
  • 33
  • 63
0
votes
1 answer

REST API for streams redeployment

Is it possible to redeploy streams using the REST API? The current documentation does not provide much info - https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#api-guide-resources-stream-deployments I am guessing one…
Abhishek
  • 1,175
  • 1
  • 11
  • 21
0
votes
1 answer

Implementing Avro in Spring Cloud Dataflow

I am trying to implement a Spring Cloud Dataflow stream that reads records from a database, passes these to a processor which converts into a Avro schema and then pass this to be consumed by a sink application. I have the data flowing from the SQL…
Bah91
  • 23
  • 4
0
votes
0 answers

Spring cloud composed task event

I'm after the composed task execution listener, that publishes events to middle-ware, basically, the same behavior as documented for custom task here Is there any way to enable this feature for composed tasks run via SCDF REST API ? Thanks
Alexander.Furer
  • 1,817
  • 1
  • 16
  • 24
0
votes
1 answer

Failed steps are not recorded in DB

I'm simulating the error handling behavior of composed task, the failedTask composed task definition looks like this : The sampleTask class runner : public class…
Alexander.Furer
  • 1,817
  • 1
  • 16
  • 24
0
votes
0 answers

Spring cloud dataflow batch tasks are logging all the arguments

We are trying to migrate from spring xd to spring cloud data flow. We are able to schedule batch job using spring cloud data flow. But one thing I've noticed is, in addition to the job arguments supplied, the framework is also logging framework…
ssm75
  • 1
  • 3
0
votes
1 answer

Content based routing using Spring Cloud Dataflow

I need to achieve content based routing and filtering using Spring Cloud Dataflow. For e.g. If the input to my processor is a String. If the String contains say xyz I need to pass it to component/step X in my Spring Cloud Stream; otherwise I want…
Kanika Sharma
  • 11
  • 3
0
votes
1 answer

Launching composed task built by DSL from stream application

Every example I've seen (task-launcher sink and triggertask source ) shows how to launch the task defined by uri attribute. My tasks definitions look like this : sampleTask sampleTask-t1 timestamp …
Alexander.Furer
  • 1,817
  • 1
  • 16
  • 24
0
votes
1 answer

Error starting SCDF plugin in Ambari

I installed SCDF using Ambari, but I cannot seem to successfully start Spring Cloud Dataflow server. I am getting the following output from the log: 03:07:30.592 [main] ERROR org.springframework.boot.SpringApplication - Application startup…
user1785917
  • 113
  • 2
  • 8
0
votes
1 answer

Spring clould data flow and spring clould task batch jobs

We have been using spring batch for below use cases Read data from file, process and write to target database (batch kicks off when file arrives) Read data from remote database, process and write to target database (runs on scheduled interval,…
Nitty
  • 41
  • 1
  • 5
0
votes
3 answers

How to remote debug a Spring Cloud Data flow Task

We are using Spring XD for executing some batch jobs and considering to use Spring Cloud Dataflow. For this I wanted to remote debug a execution of a Task and I was not able to make it working. I tried to export the following environment variable…
ssm75
  • 1
  • 3
0
votes
1 answer

Spring Cloud Dataflow local deployer properties

I am using the Spring Cloud Dataflow local server/deployer and upgrading from 1.1.4 to 1.2.3. I've found my stream configs such as value.deserializer are not being passed into the sink process correctly. I have tried…
Kevin Niemann
  • 51
  • 8
0
votes
0 answers

Deployment issue for custom app in spring cloud data flow

While deploying stream I am getting below error and deployment is failed java.lang.RuntimeException: Exception trying to list configuration properties for application…
Rohan Surve
  • 1
1 2 3
…
84 85