Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
1 answer

Spring cloud data flow deployment

I wanna deploy the Spring-cloud-data-flow on several hosts. I will deploy the server of Spring-cloud-data-flow on one host-A, and deploy the agents on the other hosts(These hosts are in charge of executing the tasks). Except the host-A, all the…
Mk.Wang
  • 11
0
votes
1 answer

Spring Boot Data Flow : Page Size must not be less than one

I am trying to register App to Spring Boot Data Flow local Server. But I am getting following error : on the dashboard. Page Size must not be less than one How to fix this issue? Am I missing any dependency? However, the app is successfully…
Forkmohit
  • 733
  • 3
  • 12
  • 31
0
votes
1 answer

Spring Cloud App Starter, sftp source, recurse a directory for files

I am using SFTP Source in Spring cloud dataflow and it is working for getting files define in sftp:remote-dir:/home/someone/source , Now I have a many subfolders under the remote-dir and I want to recursively get all the files under this directory…
gamepop
  • 424
  • 2
  • 13
0
votes
1 answer

How do I get the Spring Cloud Stream App Starter httpclient processor to send error messages from an api response to a log sink?

When I check the stdout logs for my httpclient processor, I see that I'm getting a 422 response when I make requests. The actual error message sent with the response is lost, or at least I'm not sure how to get to it. Is it possible to send the json…
James Hines
  • 341
  • 1
  • 4
  • 12
0
votes
1 answer

How to set -Xmx for Spring CDF app running on Kubernetes?

According to documentation I should be able to set -Xmx of an application with the deployer.time.memory property. I created the following stream definition: dataflow:>stream create --name ticktock --definition "time | log" And I've tried deploying…
alturkovic
  • 990
  • 8
  • 31
0
votes
1 answer

DLX (rabbitmq) routing key issue with spring cloud dataflow

I am endeavouring to upgrade to the latest spring cloud/dataflow/stream release in order to acquire the recent addition of being able to specify the delivery mode persistence on messages that end up being dead-lettered in a dataflow stream. I tried…
bikerlad
  • 13
  • 5
0
votes
1 answer

How do I use a SpEL expression on the aggregator.aggregation option of the Spring Cloud Stream Aggregator Processor?

I have three separate message payloads: "id": "1234" "type": "space-crite" "attributes": { "ferocity": 10, "appetite": 10, "cuddliness": 0 } I would like to use the Spring Cloud Stream aggregator-app-starter processor to combine these three…
James Hines
  • 341
  • 1
  • 4
  • 12
0
votes
1 answer

Input Channel subscriber not up for Custom Sink in Spring Cloud Data flow

I was trying to deploy my own custom sink of spring cloud data flow onto cloud foundry. My Dependency are below : org.springframework.cloud.stream.app …
rahul
  • 71
  • 1
  • 1
  • 7
0
votes
1 answer

Spring Cloud Stream: how to pass deployment properties to specific applications when they have identical names

Simple example from app starters: stream create --name fileTofile --definition 'file --file.directory=./input | file --file.directory=./output' Next, when deploying the stream I would like to override "file.directory" property for one of the apps.…
rideronthestorm
  • 727
  • 1
  • 13
  • 32
0
votes
1 answer

How SCDF stream diagram reflect multiple source channel?

If a have a source app(named A-Source) which has multiple channels to emit messages, eg. channelA.destination=b-topic, channelB.destination=c-topic. The receiver for b-topic is B-Sink, for c-topic is C-Sink. How can i construct my stream, …
zjunothing
  • 61
  • 3
0
votes
1 answer

properties not work inside spring-cloud-stream application when using spring-cloud-dataflow

I'am using spring-cloud-data-flow-server-yarn and successfully deployed my spring-cloud-stream apps in my yarn-cluster. Now I'm facing a problem: When I deploy my scs apps, the property value of spring.cloud.stream.bindings.output.destination will…
zjunothing
  • 61
  • 3
0
votes
1 answer

App properties in Spring Cloud Data Flow application

Based on the documentation for Spring Cloud Data Flow (SCDF) only properties that are prefixed by either "deployed." or "app." are considered when deploying an application (be it a source, processor or sink) as part of a stream. However, I've…
Enrique Medina
  • 197
  • 1
  • 12
0
votes
1 answer

Custom SCDF Starter App writes to "output" topic

I'm developing a custom starter app using the code here, because I wanted to add TcpSSLContextSupport to it. I've managed to do that, and it worked fine on my local SCDF server, but now I want to deploy it on SCDF for Kubernetes over Google Cloud.…
ystark
  • 565
  • 9
  • 18
0
votes
1 answer

Is there any way to implement jdbc user authentication in Spring Cloud Dataflow?

This is My Code that I have written and i used my Custom User Detail Service here but every time im trying to run this code bean not found expection is comming i Want to know is there any way to implement it because i saw auth object has one method…
Subhadip Majumder
  • 323
  • 1
  • 3
  • 11
0
votes
2 answers

Managing Spring Cloud Dataflow Server logs

I am deploying the spring cloud dataflow server using docker. I have created a data processing pipeline inside the dataflow server by deploying couple of spring boot application as source, processor and sink. In order to access the log of each…
krajwade
  • 133
  • 1
  • 12
1 2 3
…
84 85