Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
1 answer

Is Spring Cloud Dataflow local server able to be distributed?

I have been using SCDF for a while, and realise the main diffs between XD and SCDF is XD is born to be distributed, but SCDF seems work like a platform for SC stream apps. At least local server works like so. So my question is, is it possible that…
cinqS
  • 1,175
  • 4
  • 12
  • 32
0
votes
1 answer

Spring cloud dataflow app http returns 401 unauthorized

I am testing scdf with iris PA example which deployed "http | pmml | log" example But when I post a request to the server, I always got: 401 Unauthorized like these: { "timestamp": 1484718462061, "status": 401, "error": "Unauthorized", …
cinqS
  • 1,175
  • 4
  • 12
  • 32
0
votes
1 answer

Spring Cloud Data Flow aggregator example

Can anybody point to some examples specific to String Cloud Data Flow (not Stream XD) on how to write splitters and aggregators? In particular non trivial splitting and joining of messages. Any non trivial example of complex streams. How do you…
ka2
  • 151
  • 2
  • 12
0
votes
3 answers

PCFDev Spring Cloud Data Server app import failes

I have installed PCFDev in a corporate environment behind proxy. All proxy environments are setup and able to get to maven repos from commandline. in SCDF server however doesn't seem to reach to maven repo. System provided environment variables are…
gamepop
  • 424
  • 2
  • 13
0
votes
1 answer

Spring Cloud DataFlow - How to use a custom TCP encoder/decoder in TCP source

I have already asked this question for Spring XD. I am now trying to migrate to Spring CDF. I found this link and I tried to reuse the code there and change the encoding with mine. I created the following POM:
alturkovic
  • 990
  • 8
  • 31
0
votes
1 answer

Spring cloud dataflow DestinationReslutionException when running sample source

I have problem with running sample spring-cloud-dataflow source app. Application exits with code 0 (and it should be working). I debugged it and noticed following exception being thrown: DestinationResolutionException: failed to look up…
rideronthestorm
  • 727
  • 1
  • 13
  • 32
0
votes
1 answer

How to properly create Spring Cloud Task with custom parameters?

According to the samples here (actually - timestamp task), I have implemented a small task class: @SpringBootApplication @EnableTask @EnableConfigurationProperties({ RestProcessorTaskProperties.class }) public class RestProcessorTaskApplication { …
Mikhail Gordienko
  • 119
  • 2
  • 10
0
votes
1 answer

Deploy to YARN with Spring Cloud Dataflow

When deploying a stream to a remote YARN cluster, I get the following error from the YARN UI: Diagnostics: File file:///dataflow/apps/stream/app/application.properties does not exist This file exists on the Dataflow server' side, and contains the…
Alexandre FILLATRE
  • 1,305
  • 11
  • 20
0
votes
1 answer

How to deploy autonomous application with Spring DataFlow?

My application is configured to read a topic from a configured Kafka, then write the transformed result in the Hadoop HDFS. In order to do so, it needs to be launched on a Yarn cluster node. In order to do so, we'd like to use Spring DataFlow. But…
Alexandre FILLATRE
  • 1,305
  • 11
  • 20
0
votes
2 answers

Spring Dataflow and Yarn: How to set properties properly?

How can one change the default appdeployappmaster properties ? When I'm trying to deploy an application through Spring DataFlow YARN. I registered my app, created a stream, and click the "deploy" button. When doing so, I get the following error…
Alexandre FILLATRE
  • 1,305
  • 11
  • 20
0
votes
1 answer

Can a Spring Cloud Stream in Spring Cloud Data Flow be triggered?

We have a scenario wherein we have to process each entry of data in a database table. But this should be user triggered when they will decide to start the processing. We are happy with the stream processing since we could decouple each process…
J.D.
  • 23
  • 2
0
votes
1 answer

Recursion in spring-cloud-dataflow

Can I use spring-cloud-dataflow(kubernetes) for recursive tasks? I need to create new instance of flow depending on current results(recursive). I need distributed processing of tasks. 1) Do I need to use message-source? As I unerstand, dataflow…
Vladimir
  • 1
0
votes
0 answers

spring cloud dataflow source file | sink gpfdist speed?

could somebody help me with these. I got very big files (csv format with 5 columns) aprox 500Mb-1Gb wich i need to insert in greenplum database. I use source file to read these files with option --mode=lines and sink gpfdist to import these data in…
Berislav Purgar
  • 11
  • 2
0
votes
1 answer

Aggregate Messages in Spring Cloud Stream

I am new to spring cloud and looking to change our mono structure to micro services , this said at first what i am trying to do now is the following Receive Requests to invoke a web service (External System) from different sources. At any certain…
Shahbour
  • 1,323
  • 1
  • 16
  • 42
0
votes
2 answers

Spring Cloud DataFlow Rabbit Source: how to intercept and enrich messages in a Source

I have been successfully evaluating Spring Cloud DataFlow with a typically simple flow: source | processor | sink. For deployment there will be multiple sources feeding into this pipeline which I can do using data flow labels. All well and…
bikerlad
  • 13
  • 5
1 2 3
…
84 85