Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
2
votes
0 answers

Spring integration/cloud-stream/cloud-dataflow

Our application needs to read from Kafka and use Scatter-Gather/Aggregate pattern to do processing and finally writes to datastore. We are using Spring integration for couple of years and are really comfortable with it. However, we want to leverage…
Sasi
  • 89
  • 8
2
votes
3 answers

Spring Integration Flow with Jdbc Message source which has dynamic query

I am trying to do a change data capture from oracle DB using spring cloud data flow with kafka as broker. I am using polling mechanism for this. I am polling the data base with a basic select query at regular intervals to capture any updated data.…
Akhil Ghatiki
  • 1,140
  • 12
  • 29
2
votes
1 answer

Spring Cloud Dataflow Custom App stuck in Deploying state

I created a custom Spring Cloud Stream Processor Application and deployed it as the processor step in a Source|Processor|Sink stream. Everything seems to be working fine, but my custom app shows "deploying" in the dataflow UI. I'm deploying it as a…
brub
  • 1,083
  • 8
  • 18
2
votes
1 answer

Why is the constructor method being called before setup

Here is my class under test: KafkaProcessorApplication @EnableBinding(Processor.class) @EnableConfigurationProperties(KafkaProperties.class) public class KafkaProcessorApplication { @Autowired private Processor processor; @Autowired …
KumarAnkit
  • 713
  • 1
  • 9
  • 26
2
votes
1 answer

Spring Cloud Stream App VS Spring Cloud Task App

I'm exploring the the SCDF for orchestrating the existing batch task for my system. I noticed that the SC Stream App Starters already provides some OOTB ready to use streaming applications, which suits my use case of: Extracting xml file from FTP…
mengjiann
  • 275
  • 3
  • 12
2
votes
2 answers

Spring Cloud Data Flow : Cannot run program "docker"

I want to deploy Spring Boot applications using Kinesis streams on Kubernetes cluster on AWS. I used kops in an AWS EC2 (Amazon Linux) instance to create my cluster and deploy it using terraform. I installed Spring Cloud Data Flow for Kubernetes…
Florent Gornes
  • 198
  • 1
  • 6
2
votes
2 answers

Spring cloud data flow wirh

Docker Compose Customization - as per the reference guide if we point to mysql in the dockercompose.yml, will that start the mysql data base process, along with other processes kafka, zookeeper, and dataflowserver, or do we need to first manually…
Deepali Bhandari
  • 83
  • 1
  • 8
2
votes
1 answer

Error when launching tasks with multiple arguments from UI spring cloud dataflow

I have a batch job with takes 2 parameters. when I pass these arguments while running the jar file directly, it works. Also when i pass these arguments via the shell of the spring cloud dataflow it works: dataflow:>task launch --name adv-load…
Siddhant Sorann
  • 313
  • 3
  • 15
2
votes
1 answer

Registering Custom Spring Cloud Task with Spring Cloud Data Flow

I'm getting started with Spring Cloud Data Flow and want to implement a simple Spring Cloud Task I want to use with it. I created a hello world example from the documentation. When I run it in my IDE it executes without any problems and prints…
user6845507
2
votes
1 answer

Partition Count override

From here https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/blob/master/spring-cloud-stream-binder-kafka/src/main/java/org/springframework/cloud/stream/binder/kafka/KafkaMessageChannelBinder.java it seems is not possible, but asking…
ka2
  • 151
  • 2
  • 12
2
votes
3 answers

Spring cloud data flow shell : Stuck on "The stream is being deployed"

I successfully registered three apps named appSink, appSource and appProcessor as follows dataflow:>app register --name appSource --type source --uri maven://com.example:source:jar:0.0.1-SNAPSHOT --force Successfully registered application…
frigocat
  • 283
  • 2
  • 13
2
votes
0 answers

How to create complex POJO from multiple lines with Spring Batch

I'm new to Spring Batch and I'm kinda lost with a need I have to fulfill with this batch job and I was wondering if any of you could enlighten me about how can I properly do things right in this case. My need here is basically receive a text file…
Enrico Bergamo
  • 163
  • 1
  • 14
2
votes
1 answer

Build error when upgrade spring-cloud-starter-dataflow-server-local to 1.3.0

Previously I was using spring-cloud-starter-dataflow-server-local 1.2.3.RELEASE and it worked perfectly. When I upgrade to 1.3.0.RELEASE, the build failed with the compilation error: org.springframework.beans.factory.BeanCreationException: Error…
Michael
  • 103
  • 2
  • 11
2
votes
2 answers

Spring profile with Spring Cloud Data Flow

How to set spring.active.profiles when launching task (from shell or dashboard)? I always have "No active profile set, falling back to default profiles: default". This is because I don't know how to set VM arguments like…
bdeweer
  • 135
  • 1
  • 14
2
votes
2 answers

Spring Cloud Data Flow Edit Existing Streams

In Spring Cloud Data Flow (1.3.0) I can create and destroy streams, but how can I edit existing streams? E.g. if I want to temporarily route something to a log, and then later remove the logging, do I have to destroy and then re-create the streams?…
Jerry
  • 1,127
  • 2
  • 17
  • 35
1 2 3
…
84 85