Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
3
votes
1 answer

How to set scheduler for Spring Batch jobs in Spring Cloud Data Flow?

I’m setting up a new Spring Batch Jobs and want to deploy it using SCDF. However, I have found that SCDF does not support scheduler feature in local framework. I have 3 questions to ask you: Can someone explain how scheduler of SCDF work? Are…
Zach Pham
  • 280
  • 4
  • 18
3
votes
1 answer

java.lang.IllegalStateException when binding mysql service to skipper server in PCF

I have pushed the scdf server jar and skipper server in PCF, now I want to change the existing DB i.e. H2 to MySQL, For this I have created the MySQL service from marketplace , but when I tried to bind the MySQL service to skipper server app I am…
Pravin
  • 45
  • 1
  • 4
3
votes
1 answer

Spring Cloud Dataflow - handling argument in Task

I would like to pass the ID of an object to a variable when starting a Task in Spring Cloud Dataflow. I know that it can be done with arguments or parameters, but I don't know how to handle these arguments or parameters in Java code so I can take…
xampo
  • 369
  • 1
  • 7
  • 22
3
votes
1 answer

Bottle-necking in Stream Apps Causes Lost Messages

Spring Cloud Data Flow (SCDF) Server for Cloud Foundry v1.4.x RabbitMQ service tile provisioned for message transport A deployed spring cloud data flow stream has a processor that can produce outgoing messages more quickly than a downstream…
Channing Jackson
  • 163
  • 10
3
votes
1 answer

spring boot batch to spring cloud task with multiple jobs

I have a spring boot batch application that has 5 unique jobs that execute by console using the command: java -jar artifactName jobName param1 but now this project will be move to cloud, so I need to use spring cloud task. So far so good. I know…
Rodrigo Caceres
  • 31
  • 3
3
votes
1 answer

Spring cloud task on data flow server vs @EnableTask

I am new to Spring cloud data flow. Trying to figure out what it mean by registering, creating a task on spring cloud data flow server vs running a spring class with annotation @EnableTask Any clarification to understand what these two are, will be…
mjs
  • 139
  • 8
3
votes
2 answers

Kubernetes : Service Accounts Permissions issue with Spring Cloud Data Flow Server

I have been trying to setup Spring Cloud Dataflow Server for Kubernetes locally using minikube. Have followed the installation instructions in the the link here : SCDF Installation Reference I've been getting the below error for the SCDF…
mukulSharma
  • 113
  • 2
  • 6
3
votes
1 answer

NoClassDefFoundError while trying to run jar

My code compiles fine but when I try to run my Jar, I get the following error in the stack trace java.lang.IllegalStateException: Error processing condition on org.springframework.cloud.stream.config.codec.kryo.KryoCodecAutoConfiguration Caused by:…
frigocat
  • 283
  • 2
  • 13
3
votes
0 answers

Spring Batch/Task Repository Data Mismatch when introducing use of Tasks after historic Spring Batch data already exists

My project has 20+ batch jobs that are built with Spring Batch and have been in Production for a couple of years. We are currently in the process of migrating them to individual Spring Boot applications that are built with Spring Batch and Spring…
lframirez89
  • 41
  • 4
3
votes
1 answer

Building a File Polling/Ingest Task with Spring Batch and Spring Cloud Data Flow

We are planning to create a new processing mechanism which consists of listening to a few directories e.g: /opt/dir1, /opt/dirN and for each document create in these directories, start a routine to process, persist it's registries in a database (via…
Enrico Bergamo
  • 163
  • 1
  • 14
3
votes
0 answers

Kafka ItemWriter for Spring Batch for Spring Cloud Data Flow

I've created a Spring Batch application which reads data from file and write to database. I want to write output data into Kafka. I've tried to use Spring Cloud Data Flow but could not even run it. I've followed that tutorial:…
kamaci
  • 72,915
  • 69
  • 228
  • 366
3
votes
1 answer

Create Stream with one source, two parallel processors and one sink in Spring Cloud Data Flow

I am trying to create a stream in Spring Cloud Data Flow with One source i.e. order-source and Order message will be published to the RabbitMQ Topic/Queue. Two parallel processors i.e. product-processor and shipment-processor Both of these…
David
  • 504
  • 5
  • 21
3
votes
1 answer

How to build Spring Cloud Stream JMS ActiveMQ

I recently discovered that Spring has an alpha version of a spring cloud stream provider that leverages jms (ActiveMQ virtual destinations under the hood). This is absolutely fascinating and I want to test it out. I am having difficulty finding a…
Matthew Fontana
  • 3,790
  • 2
  • 30
  • 50
3
votes
1 answer

Spring Cloud Data Flow: task can't start on Kubernetes cluster

I've deployed Spring Cloud Data Flow server on a local Kubernetes cluster. All seems fine. Then I create an App of type Task, giving the URL of a Spring-Boot JAR. Then I create a task 'definition' and launch it. The task definition hangs in…
Sylvain Charton
  • 31
  • 1
3
votes
1 answer

Sink component doesn't get the right data with kafka in spring cloud data flow

I am not a native English speaker but I try to express my question as clear as possible. I encountered this problem which has confused me for two days and I still can't find the solution. I have built a stream which will run in the Spring Could Data…
Alex Xu
  • 33
  • 2
1 2
3
…
84 85