Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
2 answers

Spring Cloud Stream header enricher circular reference and NPE

Using Spring Cloud Data Flow Server and shell 1.3.0.M1, the Kafka binder, and loading apps from the Bacon bit-ly URL (1.2.0.RELEASE). I created/deployed a stream in SCDF shell: source | httpclient | header-enricher --headers=\"key=payload\" |…
user944849
  • 14,524
  • 2
  • 61
  • 83
0
votes
1 answer

Spring Cloud Dataflow server - deployed streams ended up in unknown state

Currently I am using spring-cloud-dataflow-local server with postgres database. I am using spring-cloud-stream applications for importing data from different sources, transform the data and finally store data in the postgres database. I have…
gvk
  • 156
  • 1
  • 8
0
votes
2 answers

Understanding Spring Cloud Stream content types

I wrote a custom processor similar to this one. In particular, the processor takes an InputDto and returns json. Following guidance in the Q&A, my custom processor has an application.properties file with this…
user944849
  • 14,524
  • 2
  • 61
  • 83
0
votes
1 answer

Not able to execute task using stream in cloud foundry

I have created a stream which is working in local and able to execute task by keeping spring-cloud-deployer-local dependency in the class path of sink application. Now the same application I want to test in PCF by changing the deployer with…
rocky
  • 737
  • 4
  • 11
  • 19
0
votes
1 answer

Spring Cloud DataFlow run Composed Task multiple times

Is there any way to tell Composed task runner (CTR) to run composed task multiple times? For now, in SCDF 1.2.3.RELEASE and CTR 1.0.0.RELEASE, if particular task in composed task has been completed, CTR doesn't execute it again.
Markiza
  • 444
  • 1
  • 5
  • 18
0
votes
1 answer

Not able to execute task using spring cloud stream

I have three application as source , processor and sink. Source is able to pass message to processor but processor is not able to send message to sink and throwing exception. Trying is execute task using spring cloud stream locally, so I have…
rocky
  • 737
  • 4
  • 11
  • 19
0
votes
1 answer

Debugging Spring Cloud Dataflow apps

I am fairly new to Spring so this might be a bit of a remedial question, but how does one attach a debugger to a custom SCDF stream app? I have found plenty of resources on how to do a remote debugging session for standard Spring apps using…
Lane Goolsby
  • 594
  • 1
  • 8
  • 26
0
votes
1 answer

Spring Cloud DataFlow Composed Task doesn't start

I'm trying to run SCDF Composed Task. Any composed tasks that I create does the same thing: executes SQL script: Executing SQL script from class path resource [org/springframework/cloud/task/schema-h2.sql]` And on that point execution pauses. In…
Markiza
  • 444
  • 1
  • 5
  • 18
0
votes
0 answers

spring cloud dataflow http - enforce SSL

I have SCDF deployed on PCF environment (1.2.3 Release) I need to enforce communication only for trusted parties for HTTP endpoint; On the first approach - I have installed 3rd party certificates on the PCF level; SSL connection is stablished…
Wojtek Jacko
  • 1
  • 1
0
votes
1 answer

Spring Cloud Dataflow Maven Snapshots - how to set update policy?

I'm developing a custom Spring Cloud Stream Processor and using a local Spring Cloud Dataflow Server (v 1.3.0.M1) to run it. As I'm still experimenting, I am using Maven to deploy snapshots to our internal remote repository, then registering them…
user944849
  • 14,524
  • 2
  • 61
  • 83
0
votes
1 answer

Spring Cloud DataFlow automated run

I want my application to run DataFlow task after some condition is satisfied (some event raises). Is there any way to run that from code: Some event raised Spring DataFlow starts some task without Spring DataFlow shell, just start all from my…
Markiza
  • 444
  • 1
  • 5
  • 18
0
votes
1 answer

Spring Task in Spring Cloud Dataflow on PCF can't find java

i have a Spring Cloud Task fat jar that i have successfully deployed to SCDF running on PCF. i have created a definition for it and can therefore run it from the dashboard. fwiw it reads and writes from a database using Spring JDBC. i'm trying to…
Wings_Of_Fire Equinox_AJ
  • 1
  • 2
0
votes
2 answers

Configure Spring Data Flow with RabbitMQ - Connection refused (Connection refused)

I am trying to run spring dataflow local server and configure rabbitmq binder for all applications. I have rabbitmq running on docker on host 172.17.0.2. java -jar spring-cloud-dataflow-server-local-1.2.3.RELEASE.jar…
moa
  • 83
  • 1
  • 2
  • 7
0
votes
1 answer

Spring Cloud Dataflow: Rabbit Source | Transform to GPB | Log Sink

Right now, I am getting my feet wet using Spring Cloud Dataflow and I have a use case where we would like to record traffic on the RabbitMQ Message Bus to a Log file. My first attempt was a simple: create --name test --definition "rabbit | log"…
Dennis Jansky
  • 107
  • 12
0
votes
2 answers

Dockerizing a stream app for SCDF

I have been able to set up spring cloud dataflow 1.2.0.RELEASE in a Kubernetes cluster, import the starter apps, and run simple flows such as "http | log". But when I try to run my own Dockerized stream app in a stream like "http | myApp | log", I…
E. Yuan
  • 1
  • 1
1 2 3
…
84 85