Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
1 answer

Spring Cloud components confusion

How do these Spring components relate/differ to/from each other? What does each represent conceptually? Would one use them together or are they competing projects? Spring Cloud Data Flow Spring Cloud Stream Spring Cloud Task Spring Cloud Task App…
Hilikus
  • 9,954
  • 14
  • 65
  • 118
0
votes
1 answer

multiple user management on SCDF

I am working on an academic project where i need to provide SCDF as a SaaS. So different users should be able to connect to their SCDF and access, deploy, and manage their streams and tasks. Of course, other users shouldn't access those private…
soulei1990
  • 1
  • 1
0
votes
1 answer

Spring Cloud Task for functional programming - how really to achieve it?

We have a need to develop on-demand applications on the lines of FRP (something similar to AWS Lambda) on our private Cloud Foundry stack. This is in order to save cost on otherwise always running low volume applications. The need is to…
Ashwin Gupta
  • 920
  • 1
  • 11
  • 17
0
votes
1 answer

How can I define a new stream by REST call on Spring Cloud Data Flow?

I would like to define and deploy a new stream on spring cloud data flow using the REST API. But on documentations, there is no reference about how I need to structure my "POST" call. When I send a POST call to…
Lucas
  • 3
  • 2
0
votes
2 answers

Unsuccessful deployment on Spring Cloud Dataflow with Apache YARN

I have installed a single-node Apache YARN with Kafka-Zookeeper and Spring Cloud Dataflow 1.0.3. All is working fine, but when I made some deployment examples, like: stream create --name "ticktock" --definition "time | hdfs --rollover=100"…
Juan Echeverria
  • 1
  • 2
0
votes
1 answer

What could cause a Spring Cloud Dataflow app to be Running but not Ready on Kubernetes?

I've set up a little Kubernetes cluster using Kubeadm and deployed the SCDF Kubernetes Server on it. When I connect using the Shell and deploy a stream, the apps become Running, but the Ready column shows 0/1 rather than 1/1. Kubernetes then…
Remon Sinnema
  • 51
  • 1
  • 10
0
votes
1 answer

Spring Cloud DataFlow shell command "not currently available"

I sometimes get this error: 2016-10-20T15:38:24+0200 WARN main o.s.s.c.SimpleParser:131 - Command 'app register --name email --type source --uri ' was found but is not currently available (type 'help' then ENTER to learn about…
Remon Sinnema
  • 51
  • 1
  • 10
0
votes
1 answer

How To Set an local Maven Repository (Spring Cloudataflow Server) -->Pivotal CF DEV

Environment Pivotal Cloud Foundry DEV Spring Cloud Data Flow Server Spring Cloud Data Flow Shell Maven Specific Environment Variables (Spring Cloud Data Flow Server) MAVEN_LOCALREPOSITORY…
Sheep
  • 23
  • 5
0
votes
1 answer

Dependence on External Maven and Git Resources

Application information: Spring Cloud Data Flow Server Cloudfoundry 1.0.0.RELEASE (DIY built with Spring Cloud Config Server dependencies) Spring Cloud Config Server PCF Elastic Runtime 1.7.x I'm curious about the extent to which applications…
Channing Jackson
  • 163
  • 10
0
votes
1 answer

Is there a way where we can set system environment variables separately for each App while deploying the stream

We are using a third part security library which requires a system environment variable to be set. Can we pass as parameter in stream definition or while deploying stream separately for each app?
S C
  • 1
0
votes
1 answer

Is there a commercially supported option for a standalone Spring Cloud Data Flow?

We're looking at using Spring Cloud Task / Spring Cloud Data Flow for our batch processing needs as we're modernising from a legacy system. We don't want or need the whole microservices offering ... we want to be able to deploy jobs/tasks, kick off…
Phil Cairns
  • 758
  • 4
  • 15
0
votes
2 answers

Spring Cloud Data Flow DLQ Configuration not working

I am trying to configure DLQ in Spring Cloud Data Flow. Here is the Stream defination and how I am deploying it stream create --definition ":someTestTopic > custom-transform --spring.cloud.stream.bindings.input.consumer.headerMode=raw |…
Viren Balaut
  • 55
  • 2
  • 6
0
votes
1 answer

How to know when a file has been sunk on HDFS using Spring Cloud Dataflow

I'm downloading a file source and creating and stream to process line by line to finally sink into HDFS. For that purpose I'm using Spring Cloud Dataflow + Kafka. Question: is there any way to know when the complete file has been sunk into HDFS to…
pedvaljim
  • 108
  • 7
0
votes
1 answer

Spring Cloud Dataflow routing by JSON header

I have been trying to create a stream with Spring Cloud Dataflow but have not had much luck (mostly due to the lack of documentation). Issue 1: Accessing web GUI of dockerized Spring Cloud Dataflow I have a dockerized Spring Cloud server running…
user2990276
  • 307
  • 1
  • 2
  • 7
0
votes
2 answers

setting env var values in deployed stream app

Using the following: Spring Cloud Data Flow Server Cloudfoundry 1.0.0.RC1 Spring Cloud Config Server service I rebuilt the spring-cloud-dataflow-server-cloudfoundry with the additional dependency to enable its binding with Spring Cloud Config…
Channing Jackson
  • 163
  • 10
1 2 3
…
84 85