Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
1 answer

Spring Cloud Data Flow and SCDF Kafka DataSource

I'm following the getting started guide at [1], but I've removed the MySQL and analytics stuff from the config settings since I do not intend to use any analytics functions. The scdf service however crashes later, saying that no data source is…
benjist
  • 2,740
  • 3
  • 31
  • 58
0
votes
1 answer

Can Spring Cloud Task's Partitioned Job be executed in Spring Cloud Data Flow?

I am trying to setup and execute the Spring Cloud Tasks Sample of partitioned batch job (https://github.com/spring-cloud/spring-cloud-task/tree/master/spring-cloud-task-samples/partitioned-batch-job) in Spring Cloud Data Flow Server. But for some…
J.D.
  • 23
  • 2
0
votes
3 answers

Setting up Spring Cloud Data Flow on Kubernetes

Do I need to install an instance of Spring Cloud Data Flow on the master server myself, or is this getting installed "automatically" as part of the deployment? This isn't quite clear from the description at…
benjist
  • 2,740
  • 3
  • 31
  • 58
0
votes
1 answer

Options/properties for apps not showing on data flow server or shell

When attempting to create a new stream in the data flow server or shell there are no options/properties being shown. Similar output below from shell. I've also include the logs from the SCDF server -…
Paul Barriscale
  • 21
  • 6
0
votes
2 answers

Spring cloud stream - Kafka binder performance

I have the following simple piece of code: private int i = 0; @StreamListener(Sink.INPUT) public void processMessage(Message message) { i++; } @Scheduled(fixedDelay=5000) private void scheduled(){ …
ltalhouarne
  • 4,586
  • 2
  • 22
  • 31
0
votes
1 answer

Ready to run docker images for Spring Cloud Data Flow

I'm looking for ready to run and ready to deploy images for Spring Cloud Data Flow, optimally with HDFS Fuse file system driver. Preferrably with example and documentation. I already looked for some tutorials - but there are so many steps and…
benjist
  • 2,740
  • 3
  • 31
  • 58
0
votes
1 answer

httpclient processor `replyExpression` variables

I'm building the pipeline with spring cloud stream starters and httpclientprocessor is one of the steps. My requirement is to invoke the remote http server and enrich the original message with the response. The problem is that httpclientprocessor…
Alexander.Furer
  • 1,817
  • 1
  • 16
  • 24
0
votes
1 answer

Spring Cloud Data flow Server Failover

What's the best practice to set-up SCDF server for Failover? I am talking about SCDF server itself. Not the stream and Tasks that you deploy in SCDF. I am planning to use Kubernetes as the runtime.
Sanjay Parmar
  • 67
  • 1
  • 1
  • 7
0
votes
1 answer

spring-cloud-stream: Stream deployments fail on spring-cloud-dataflow-server-cf

In our use-case we need to create and deploy a Message Stream application to our private Cloud Foundry environment. The stream application is meant to listen on a RabbitMq queue and trigger a task, as defined in the stream. To achieve the same, we…
Ashwin Gupta
  • 920
  • 1
  • 11
  • 17
0
votes
2 answers

Custom jdbc driver for jdbc-source-XXX application

I'm trying to use MS SQL driver with jdbc app with local data flow server jdbc --url='jdbc:sqlserver://server' --driver-class-name=com.microsoft.sqlserver.jdbc.SQLServerDriver As expected, the driver is not in classpath and I get Cannot load…
Alexander.Furer
  • 1,817
  • 1
  • 16
  • 24
0
votes
1 answer

Spring Data Flow Yarn - unable access jarfile

I try to run simple spring batch task on Spring Cloud Data Flow for Yarn. Unfortunatelly while running it I got error message in ResourceManager UI: Application application_1473838120587_5156 failed 1 times due to AM Container for…
Ragnar
  • 55
  • 1
  • 8
0
votes
1 answer

Spring Cloud Stream on Cloud Foundry : Binding a RabbitMq source to a Stream application

I have a simple need to deploy a Stream application to our internal cloud foundry instance. The stream has a RabbitMq queue as the source of messages for it, and is defined as below for running it locally on PC: stream create myMessageStream…
Ashwin Gupta
  • 920
  • 1
  • 11
  • 17
0
votes
1 answer

Can I specify which host directory to mount when deploying an app using Spring Cloud Deployer Kubernetes?

I have an app that I deploy as part of a stream with Spring Cloud Dataflow on a Kubernetes cluster. The Docker image for the app contains a VOLUME instruction and I'd like to specify a directory on the host to mount the volume to. (This is…
Remon Sinnema
  • 51
  • 1
  • 10
0
votes
1 answer

spring-cloud-starter: stream-processor-tasklaunchrequest-transform ignores local maven settings

In my cloud stream application I've used the spring-cloud-starter-stream-processor-tasklaunchrequest-transform starter app as task-processor. The starter is locally built and installed in local Maven repository. Commands used to start my local…
Ashwin Gupta
  • 920
  • 1
  • 11
  • 17
0
votes
1 answer

SCDF yarn connection error

I have deployed spring cloud dataflow on a virtual yarn cluster. starting the server ./bin/dataflow-server-yarn executes correctly. and returns 2016-11-02 10:31:59.786 INFO 42493 --- [ main] o.s.i.endpoint.EventDrivenConsumer :…
soulei1990
  • 1
  • 1
1 2 3
…
84 85