Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
2
votes
1 answer

Quoting properties in Spring Cloud Dataflow

This seem such a stupid problem, but we're trying to define dataflows stream from the SCDF shell application, but we're running into issues with quoting. Let's say we want to define a filter with a SpEL expression to just filter out anything where…
Greg Charles
  • 1,880
  • 4
  • 20
  • 39
2
votes
1 answer

Where to set env variables for local Spring Cloud Dataflow?

For development, I'm using the local Spring Cloud Dataflow server on my Mac, though we plan to deploy to a Kubernetes cluster for integration testing and production. The SCDF docs say you can use environment variables to configure various things,…
Greg Charles
  • 1,880
  • 4
  • 20
  • 39
2
votes
1 answer

how to configure spring dataflow for spring batch

I have spring batch project I want to configure it on spring cloud dataflow I m able to register it on SCDF but on launching task my job is not running following is my configuration…
sourabh
  • 21
  • 2
2
votes
1 answer

Integrating Spring Cloud Data Flow for existing Spring Batch application

I have an exiting Spring Batch application, with a Job. I would like to monitor the Job, using Spring Cloud Data Flow. How can we integrate Spring Cloud Data Flow with my existing spring batch application.
Jagadheeswaran Mohan
  • 339
  • 1
  • 3
  • 18
2
votes
0 answers

Creating a generic Avro sink in Spring Cloud Stream / Dataflow

I am trying to create a generic receiver for Avro messages in Spring Cloud Data Flow but I am running into some bother. The setup I have currently is a processor converting my input data into an Avro message and pushing this out to the sink. I am…
Bah91
  • 23
  • 4
2
votes
1 answer

Spring Cloud Stream application web interface

is it possible to create a web interface to a spring-cloud-stream application ? Many source (file, http, ftp) send data to an application to transform it in a common format, and to reduce development i was thinking to create a web interface where…
Stefano Veloccia
  • 190
  • 12
2
votes
1 answer

way to check spring cloud stream source and sink data content

Is there any way I can check what data is there in spring cloud dataflow stream source(say some named destination ":mySource") and sink(say "log" as sink)? e.g. dataflow:>stream create --name demo --definition ":mySource>log" Here what is there in…
sumit
  • 133
  • 2
  • 10
2
votes
1 answer

Spring Batch integration with spring cloud data flow on local server to add spring admin capabilities

I have a basic spring batch app that runs on embedded Apache Tomcat in spring boot. I need to add spring admin capabilities to it. As per latest spring docs, I need to use spring cloud data flow to do this…
karthik
  • 773
  • 2
  • 11
  • 19
2
votes
2 answers

spring-data-flow task example

I'm using spring-cloud-dataflow with taskcloud module but I've some trouble to lunch a simple example in container. tiny example 6.3 writing code then I've deploy it but when I try to execute it throw me an java.lang.IllegalArgumentException:…
fbalicchia
  • 67
  • 1
  • 3
2
votes
0 answers

Why mongodb source always return the entire documents but not new added document

this is the DSL using to create mongodb source ingestion to log: stream create --name mongodb7hdfs7 --definition "mongodb120 --database=sourcedb --host=172.20.74.91 --fixed-delay=5 --cron='*/10 * * * * *' …
Aska Shen
  • 21
  • 1
2
votes
1 answer

How do I add a header to a message using the header-enricher Spring Cloud Stream App Starter?

I'm trying to add a header with a key of "order_id" and a value based on a property in the payload to my messages. I then send the result to a log sink where I can inspect the headers after the header processor. Here's the stream: stream create…
James Hines
  • 341
  • 1
  • 4
  • 12
2
votes
3 answers

How do you automate deploying Spring Data Flow?

I've got an instance of Spring Cloud Data Flow stood up with my apps and streams that I need. I even wrote scripts that work within the command shell to deploy all this stuff at once. The problem is that I have a manual step now that I need to do…
Berin Loritsch
  • 11,400
  • 4
  • 30
  • 57
2
votes
1 answer

What is the difference between workflow and dataflow?

I'm looking at Spring Cloud Date Flow, before that I watched Activiti and Camunda(this is workflow engine). And I can't understand what is the difference between these concepts as workflow and dataflow? and сan we call Spring Cloud Data Flow the…
Roman
  • 61
  • 1
  • 6
2
votes
1 answer

Activating Avro message converter in Spring Cloud Dataflow

I am implementing a stream app on Spring Cloud Dataflow. I would like to use Avro based schema registry client for serialization and schema control. My basic goal is to feed Source app with some external data, transform it to prepared avro-based…
rideronthestorm
  • 727
  • 1
  • 13
  • 32
2
votes
1 answer

Spring Cloud Dataflow Type conversion not working in processor component?

I have a processor which transforms byte[] payloads into MyClass payloads: @Slf4j @EnableBinding(Processor.class) public class MyDecoder { @ServiceActivator(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT) public MyClass…
alturkovic
  • 990
  • 8
  • 31
1 2 3
…
84 85