Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
1 answer

Spring Cloud Stream : Sink loses RabbitMQ connectivity

I see that my custom Spring cloud stream sink with log sink stream app dependency loses RabbitMQ connectivity during RabbitMQ outage, tries making a connection for 5 times and then stops its consumer. I have to manually restart the app to make it…
rahul
  • 71
  • 1
  • 1
  • 7
0
votes
1 answer

Is this Bug in Spring cloud data flow :tab create 2 additional kafka queues and additional spring boot application for bridge

For my used it create additional instance of bridge and two more kafka topic which cause latency issue for data flowing between so -> s So - source p - processor s - sink Use-case for creating data flow 1) for new record flow should so => s 2) for…
Rohan Surve
  • 1
0
votes
1 answer

Spring Cloud DataFlow: Getting payload as List

Using Spring Cloud DataFlow 1.3.0.M2 with Spring Cloud Stream Starters Celsius.M1. I have two processors. First produces a List that is supposed to be consumed by the other. Here is simplified code. // Processor…
user944849
  • 14,524
  • 2
  • 61
  • 83
0
votes
1 answer

How to change the stdout and stderr Task files directory?

In Linux, spring-cloud-dataflow local server logs for Task executions is /tmp/spring-cloud-dataflow-*. I need these task-specific directories to be created somewhere else, not /tmp. E.g. instead of…
faustool
  • 37
  • 1
  • 4
0
votes
1 answer

Composed Task Runner doesn't show execution time

I created a Composed Task Runner with below definition. SuccessTask and Notifier are simple Spring Cloud Task spring boot applications that are printing just Hello World and returning an exit code 0. When I create individual tasks I'm able to see…
Kanika Sharma
  • 11
  • 3
0
votes
2 answers

A processor receive multiple message with same payload multiple time

I'm starting a new project with 'spring-cloud-dataflow', developing a bunch of jar to fit my need. One of this is a processor to untar files coming from a file source, this application use a customized version of integration-zip with features to…
Stefano Veloccia
  • 190
  • 12
0
votes
1 answer

Spring cloud data flow unable to find file

I have created a microservice in springboot, there is folder under resource folder and then a file under this folder i,e. resource mycustomfolder myfile.txt I am creating a bean, which filed populated by myfile…
Baba
  • 311
  • 1
  • 2
  • 12
0
votes
1 answer

Spring Cloud data flow does not show Spring cloud task execution details

The Spring cloud dataflow documentation mentions When executing tasks externally (i.e. command line) and you wish for Spring Cloud Data Flow to show the TaskExecutions in its UI, be sure that common datasource settings are shared among the both.…
Kanika Sharma
  • 11
  • 3
0
votes
2 answers

Analytics dashboard on Spring Cloud Data Flow does NOT recognize my stream for ANY Metric type

My Stream Definitions are as Follows:- IngestToBroker = custom-producer --spring.cloud.stream.bindings.output.content-type=application/json --spring.cloud.stream.bindings.output.producer.headerMode=raw > :Kafkatopic1 DataProcessor = :Kafkatopic1>…
Kenny Weeler
  • 153
  • 1
  • 3
  • 15
0
votes
2 answers

Spring Cloud Dataflow (rabbit | processor | hdfs) output binary

Currently, I have a use case where I need to take messages from RabbitMQ Message Bus, append the message size (in bytes) and output message using HDFS Sink. To start, I have created my own processor which appends the size to the message. The reason…
Dennis Jansky
  • 107
  • 12
0
votes
1 answer

SpelEvaluationException: EL1004E: Method call: Method getValue(java.lang.String) cannot be found on java.lang.String type

ALL I run the spring cloud dataflow with local server.when i run the stream which is defined like this http --port=8787 | transform --expression=payload.getValue('name') | jdbc --columns=name --table-name=http_table…
Shanshan Gan
  • 1
  • 1
  • 2
0
votes
1 answer

Spring Cloud Data Flow Remote RabbitMQ Server Config

I am new to SCDF and am trying to get started with a RabbitMQ transport layer and SCDF version 1.2.2. I have setup RabbitMQ in a separate VM and have the SCDF local server and SCDF shell jar in one VM. Can someone suggest how I can specify the…
Ayon
  • 315
  • 3
  • 20
0
votes
1 answer

spring cloud data flow server cloud foundry redirect to https

I have been wrestling with this for a couple of days now. I want to deploy Spring Cloud Data Flow Server for Cloud Foundry to my org's enterprise Pivotal Cloud Foundry instance. My problem is forcing all Data Flow Server web requests to TLS/HTTPS.…
Robert St. John
  • 100
  • 6
0
votes
2 answers

cloud foundry spring cloud data flow server security configuration

I am trying to deploy the Spring Cloud Data Flow Server to an enterprise Pivotal Cloud Foundry instance using an application manifest.yml. My first concern is externalizing the SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME and…
Robert St. John
  • 100
  • 6
0
votes
1 answer

Scheduling a composed task

I am trying to schedule the launch of composed task in SCDF, but have been unable to find a way to do this. The closest thing that I have found is the TriggerTask source and tasklauncher sink modules. See TriggerTask The problem with this approach…
chris_fitz
  • 23
  • 5
1 2 3
…
84 85