Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
1 answer

Wildcarding SQL query using SpringXD

I have an use-case where I need to move data from an SQL based warehouse to mongo every day. We have chosen SpringXD as a tool for this ETL job. I however need to pass in few wild carded parameters to my query which change every day. I was wondering…
Dipayan
  • 203
  • 4
  • 12
0
votes
2 answers

Spring Cloud Data Flow install on Cloud Foundry fails

I followed the instructions here http://docs.spring.io/spring-cloud-dataflow/docs/1.0.0.M1/reference/html/getting-started-deploying-spring-cloud-dataflow.html (which seems down currently), but my installation fails when trying to push the admin…
Steve Womack
  • 1
0
votes
1 answer

How to deploy streams with 'criteria' in SCDF

In this guide it is mentioned that during deployment of a stream one can specify for each module a 'criteria' property. e.g. dataflow:> stream deploy --name ticktock --properties "module.time.count=3,module.log.criteria=groups.contains('x')" The…
Alexander
  • 50
  • 5
0
votes
1 answer

Spring Cloud Dataflow on different message broker than Redis?

from what I can see in the documentation, always a Redis instance is needed for Spring Cloud Dataflow to work. Is it also possible to work with different message broker, e.g. RabbitMQ? How would one specify a different message broker during startup?
Alexander
  • 50
  • 5
0
votes
1 answer

Spring Cloud Data Flow @EnableBinding annotation breaks constructor injection

I am trying to create a bean using constructor injection and annotate it with Spring Cloud @EnableBinding annotation. I am using Spring Boot 1.3.3 and Spring Cloud Brixton.M5. I receive the following exception during app…
Max Romanovsky
  • 2,824
  • 5
  • 30
  • 37
0
votes
1 answer

spring-cloud-dataflow register custom module from local repository

I have a jar with parent spring-cloud-dataflow-parent, which I register using my local repository --coordinates. But once I execute module info on it, I receive this error: java.lang.RuntimeException:…
MattEnergy
  • 5
  • 3
-1
votes
1 answer

Connection to rabbitmq failed when deploying spring cloud data flow in the local

I referenced the documentation: https://dataflow.spring.io/docs/installation/local/manual/ to run the spring cloud data flow in my local mac, and then deployed an http | log stream according to…
Jeff Tian
  • 5,210
  • 3
  • 51
  • 71
-1
votes
2 answers

Spring Cloud Data Flow in AKS using ACR with a managed identity : metadata

We have a Spring Cloud Data Flow server (SCDF) deployed in an Azure Kubernetes Service (AKS). We defined a user managed identity assigned to this AKS. This identity has the AcrPull role on an Azure Container Registry (ACR) that doesn't have an admin…
carth
  • 1
  • 1
-1
votes
1 answer

Can we transfer file from SFTP to server through spring cloud data flow?

My requirement is to get a file from SFTP and make it available client to download. I am forcefully asked to do that using spring cloud data flow. In the documentation, I saw that there is an SFTP to the JDBC File Ingest tutorial…
Dasun
  • 602
  • 1
  • 11
  • 34
-1
votes
1 answer

Spring batch - Could not commit JDBC transaction; nested exception is java.sql.SQLTransactionRollbackException

I am using spring cloud data flow 2.7.1 with mysql 5.7.24 which executes a task with job related. the architecture used is Externalizing Batch Process Execution - remote partitioning (manger,worker) each partition is read from rabbitmq and there are…
Ivan Gallo
  • 1
  • 2
-1
votes
1 answer

Register App in spring cloud data flow from docker hub when there is no internet

How to get application Properties on registering app in spring cloud data flow from docker hub when there is no internet
Rajkumar
  • 1
-1
votes
2 answers

Can't Add JDBC driver for Oracle in Spring Cloud Data Flow

After reading the official documentation on https://docs.spring.io/spring-cloud-dataflow/docs/current-SNAPSHOT/reference/htmlsingle/#_oracle its still unclear which pom.xml the dependencies go into and where to put oracle jar file for local spring…
josh-pmi
  • 11
  • 3
-1
votes
1 answer

how to start/stop all tasks in Spring Cloud DataFlow

Does Spring Cloud Data Flow support start/stop all tasks features? (start /stop at the same time) and how to do that? Can I do that by updating from database?
Zach Pham
  • 280
  • 4
  • 18
-1
votes
1 answer

Spring cloud data flow ,jdbc-sink-rabbit source code

I only can find the source for jdbc-sink (Link), but I can't find the source code for jdbc-sink-rabbit which used in spring cloud data flow. Does anyone know it?
yp zhou
  • 3
  • 2
-1
votes
1 answer

Spring cloud data flow internal communication

Trying to use spring cloud data flow for an data pipeline where source read data from a rest stream and the sink would save it to Database.Currently i did a POC using Spring Webflux getting the streaming data and saving to DB which works fine.…
Ajith Kannan
  • 812
  • 1
  • 8
  • 30
1 2 3
…
84
85