Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
4
votes
2 answers

Configuring a Spring Cloud Data Flow Task with its own Database

I have a task application with its own database that I'd like to run in Spring Cloud Data Flow. My problem is that SCDF overwrites the datasource configuration in the task with the datasource configuration for SCDF. (Both databases are Oracle…
S0m30n3
  • 334
  • 2
  • 15
4
votes
2 answers

How to externalize application properties in Spring Cloud Data Flow taks

I currently have a regular SpringBoot App that I'm splitting into 2 applications in order to gain some of the benefits from Spring Cloud Dataflow and Spring Batch: A SpringBoot app responsible for detecting specific events and launching new Batch…
Enrico Bergamo
  • 163
  • 1
  • 14
4
votes
1 answer

Spring cloud data flow with spring batch job - scaling considerations

We are currently in evaluation process shifting from Spring batch + Batch Admin into Spring Cloud based infrastructure. our main challenges / questions: 1. As part of the monolithic design of the spring batch jobs we are fetching some general MD and…
Elad Hirsch
  • 294
  • 1
  • 16
4
votes
2 answers

Limit the total number of tasks running at same time in spring cloud data flow?

Every time we launch a spring cloud task, it starts a new jvm (java.exe), so if 25 tasks are launched , then it will start 25 jvm. I was wondering to see How to limit the total number of all tasks (running for all deployed jars) at the same time…
Raj_Twister
  • 53
  • 5
4
votes
1 answer

How to scale the spring cloud data flow server

Normally we run the Jar of spring cloud data flow in one of the machine, but what if over the period we create many flows on the machine and the server gets overloaded and becomes a single point of failure, Do we have some thing where we can run the…
shishir ranjan
  • 85
  • 1
  • 6
4
votes
2 answers

Migrating from spring-batch-admin to spring cloud dataflow

We are planning to migrate from spring-batch-admin to spring-cloud-dataflow as batch admin will be moving to into the Spring Attic with an end of life date to be December 31, 2017. As per the documentation in Migration Doc, “In the Spring Batch…
praveen
  • 43
  • 1
  • 3
4
votes
1 answer

Spring Cloud Data Flow on Amazon ECS

Are there any implementations of the Spring Cloud Data Flow Service Provider Interface for Amazon EC2 Container Service as a runtime platform? Does it make sense to consider using ECS as a runtime platform for Spring Cloud Data Flow? If there are no…
David Geary
  • 1,756
  • 2
  • 14
  • 23
4
votes
1 answer

Advantage of using Spring cloud data flow instead of spring batch

We need to write an application to read a flat file every day and write into a database table. We are planning to use Spring Batch to do this job. The limitation and addition we are looking for are 1.The application itself would run in a single VM.…
user3720735
  • 41
  • 1
  • 3
4
votes
1 answer

Kafka Source in Spring Cloud Data Flow

I am migrating from Spring XD to Spring Cloud Data Flow. When I am looking for module list I realised that some of the sources are not listed in Spring Cloud Flow - One of them is KAFKA source. My question is why KAFKA source is removed from…
Viren Balaut
  • 55
  • 2
  • 6
4
votes
1 answer

Does Spring Cloud data flow support batch processing

We are exploring various programming/library options (on Java side of the world) for faster batch processing as well as be able to be deployed on cloud. We came across Spring batch/XD/cloud data flow. From the quick review of documentation on…
JavaTec
  • 965
  • 3
  • 15
  • 30
3
votes
0 answers

Spring Cloud Data Flow global properties for Composed Tasks

I am trying to pass global properties into Composed tasks. When I specify each and every job, it works, but there is configuration duplicity. I tried multiple approaches, but the properties are not set correctly. My project (jar) contains multiple…
Tomas Lukac
  • 1,923
  • 2
  • 19
  • 37
3
votes
0 answers

Spring Batch with Spring Cloud Data Flow running out of DB connections

We have Spring Cloud Data Flow running in Kubernetes in order to orchestrate Spring Batch jobs. For each new file we have in, Spring Cloud Data Flow spins up a new Spring Batch task. Spring Batch accesses database and uses the connection pool,…
alvgarvilla
  • 1,026
  • 12
  • 25
3
votes
0 answers

Spring Cloud Dataflow aggregation of splitted list

I am trying to use SCDF to split and aggregate my list. But I can't aggregate as expected. My SCDF is being fed by a list over rabbitmq each minute. Let's say i have a list consisting of objects like: [{"foo" : "11", "bar" : "21"},{"foo" : "11",…
Selcuk
  • 857
  • 2
  • 11
  • 32
3
votes
1 answer

Need advice on how to create deploy spring-cloud-dataflow apps in several dedicated servers to create an stream

I am trying to use spring-cloud-dataflow to create streams over an existing set of servers. It is not a production environment, I am aware of the fault tolerance issues with the "local" deployer. I have in the past created this infrastructure…
ka2
  • 151
  • 2
  • 12
3
votes
0 answers

How to create simple login in Spring Cloud Data Flow?

Based on the SCDF document, as of Spring Cloud Data Flow 2.0, OAuth2 is the only mechanism for providing authentication and authorization. However, I want to create the simple login page without using Authentication provider. Could I apply Spring…
Zach Pham
  • 280
  • 4
  • 18
1
2
3
…
84 85