Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
1 answer

Dataflow 1.2.0 YAML configuration changes

Yesterday I upgraded my development environment to Spring Cloud Dataflow 1.2.0 and all of my sink/source apps dependencies. I have two main issues: javaOpts: -Xmx128m is not longer being picked up, so locally deployed apps have the default Xmx…
Kevin Niemann
  • 51
  • 8
0
votes
0 answers

Partial info stored in task repository when execute tasks

I am running a simple hello world task. I am running the spring cloud data flow server backed by a Postgres db. When I execute my task and then query the data flow server, I see: dataflow:>task execution…
Anand Rao
  • 1
0
votes
1 answer

Using logstash, config server and eureka with spring cloud task and dataflow

We have an existing microservice environment with logstash, config and eureka servers. We are now setting up a Spring Cloud Dataflow (Kubernetes) environment (primarily intially to run tasks/batch jobs). Ideally we would like the tasks to use the…
David Geary
  • 1,756
  • 2
  • 14
  • 23
0
votes
1 answer

Is it possible to use my data store in Spring Cloud Dataflow (for example, Apache Ignite or another InMemory store) for Spring Cloud Stream?

I saw in the tests spring cloud dataflow used to store the SpringDefinition - HashMap, is it possible to override the configuration of DateFlowServerConfiguration for storing streams and Tasks in an InMemory, for example in the same HashMap, if so,…
Roman
  • 61
  • 1
  • 6
0
votes
0 answers

Spring Cloud Dataflow Application Properties to Show a dropdown in Dataflow Ui

I want to know how to specify a property here so that i can get a dropdown in Spring Cloud Dataflow ui when ill register my app. I am adding screeshot of my app modal This is the property file of my app import java.nio.charset.Charset; import…
Subhadip Majumder
  • 323
  • 1
  • 3
  • 11
0
votes
1 answer

Upgrade SCDF on YARN with Ambari

I started using SCDF from spring-cloud-dataflow-server-yarn 1.0.2.RELEASE. It was installed via Ambari 2.4.2 and is now running along with HDP2.5(Hortonworks Data Platform). Now,a lot has changed in SCDF, so I want to upgrade…
JavyZheng
  • 53
  • 7
0
votes
1 answer

Spring Boot 1.5.3 local CDF server

I am trying to extend local CDF server. I am using the following dependencies: org.springframework.cloud …
alturkovic
  • 990
  • 8
  • 31
0
votes
1 answer

Migration from SpringXD to Spring cloud data flow

we are currently using SpringXD to ingest the data from the various data sources to the sinks. Since Springxd is scheduled for end of availability, we would like to migrate to Spring Cloud data flow. We have written several custom modules for…
nagendra
  • 1,885
  • 3
  • 17
  • 27
0
votes
1 answer

Spring Data Flow w/ 2 sources feeding one processor/sink

I'm looking for some advice on setting up a Spring Data Flow stream for a specific use case. My use case: I have 2 RDBMS and I need to compare the results of queries run against each. The queries should be run roughly simultaneously. Based on the…
diogenes_der_zyniker
  • 123
  • 1
  • 8
0
votes
1 answer

How can I set a path prefix for scdf UI

How can i set a path prefix for scdf UI. I already set the tomcat property (like server.contextPath=/api/) which works for the rest API but the UI is still giving errors. Info: i have it running in Kubernetes using ingress.
balistof
  • 303
  • 3
  • 10
0
votes
1 answer

LoadBalancing Spring cloud data flow server

In spring cloud dataflow, as per my understanding each stream is a microservice but the dataflow server is not. Am I right? Is it possible to have multiple instances of spring cloud dataflow(SCDF) server? How to loadbalance the dataflow server? I am…
Chandru
  • 964
  • 11
  • 16
0
votes
2 answers

Kubernetes/Spring Cloud Dataflow stream > spring.cloud.stream.bindings.output.destination is ignored by producer

I'm trying to run a "Hello, world" Spring Cloud Data Flow stream based on the very simple example explained at http://cloud.spring.io/spring-cloud-dataflow/. I'm able to create a simple source and sink and run it on my local SCDF server using Kafka,…
Daniel Gutierrez
  • 61
  • 9
0
votes
1 answer

Spring CDF not showing some properties

I have an app MyApp with MyAppProperties in one project and CommonProperties in another project. Both property classes are annotated with @ConfigurationProperties. I added META-INF/spring-configuration-metadata-whitelist.properties with the…
alturkovic
  • 990
  • 8
  • 31
0
votes
1 answer

Clarification about task executed as a source with spring cloud stream

When reading this documentation, it seems that we can write a microservice with @EnableTask that has a source whose emitted messages are recorded as an execution step by spring-cloud-task. I would like to have several clarifications about this…
Guillaume D.
  • 3,284
  • 3
  • 20
  • 26
0
votes
1 answer

how to create a topic from spring cloud data flow?

I am trying to store last read record of my source in a separate topic in kafka for a stream source. How can i achieve this with spring cloud data flow stream app. Any suggestion wud be of great help..
shelzz
  • 3
  • 2
1 2 3
…
84 85