Stack Overflow
Stack Overflow

Questions tagged [spring-cloud-dataflow]

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines. Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Use this tag for questions about the Spring Cloud Data Flow project. It is not intended for general questions about integrating other Spring projects with other technologies.

Spring Cloud Data Flow's Official Project Site

Spring Cloud Data Flow Website

Spring Cloud Data Flow's Github Repo

How to contribute

How to report issues

Related tags , , , , , .

1276 questions
0
votes
1 answer

getting error "Caused by: org.apache.kafka.common.errors.TopicExistsException: Topic "ticktock2.time" already exists"

While deploying the basic stream, gets error topic already exists Caused by: org.apache.kafka.common.errors.TopicExistsException: Topic "ticktock2.time" already exists
shelzz
  • 3
  • 2
0
votes
1 answer

Spring Cloud Data Flow app with separate sink and source

I have this application that would like to deploy using SCDF. It receives a message (sink), processes it asynchronously through a non deterministic or simple pipeline system. At the end, it might or migh not produce an output. If an output is…
ka2
  • 151
  • 2
  • 12
0
votes
1 answer

JMS bridge in Spring Cloud Dataflow

How can I setup a JMS bridge in Spring Cloud Dataflow between a topic in one server to some other topic in another server? My problem is that there is a JMS source, but I don't know how to setup the server address/username/password (I'm using Tibco…
Adrian Ber
  • 20,474
  • 12
  • 67
  • 117
0
votes
1 answer

Missing Kafka source/sink in springcloud

I would like to use Kafka Source/Sink in SpringCloud Dataflow but I cannot find thes app-starters in spring-cloud-stream-app-starters. They were available in Spring XD. Does anyone knows why they disappeared ? Thanks
LGI
  • 3
  • 1
0
votes
1 answer

How to undeploy stream after X hours in Spring CloudDataflow?

I need to deploy a stream on Spring CloudDataflow server and undeploy it automatically after 24 or 48h unless someone undeploys it earlier. Could anyone give me an idea how to approach this because I am not even sure where to begin. Any help is…
alturkovic
  • 990
  • 8
  • 31
0
votes
1 answer

How to implement LDAP user authorization in Spring-Cloud-Dataflow server

please help me with this issue. I already implemented the user authentication with spring: cloud: dataflow: security: authentication: ldap: enabled: true …
Subhadip Majumder
  • 323
  • 1
  • 3
  • 11
0
votes
1 answer

How to whitelist MongoProperties in custom processor application?

I have written a custom processor app to read data from MongoDB. In this app, I use MongoTemplate for MongoDB operation, so SpringBoot has provide the MongoProperties class for properties loading. However, when I try to white list MongoProperties so…
JavyZheng
  • 53
  • 7
0
votes
1 answer

Running multiple spring cloud task jobs within Spring Cloud data flow container on PCF

I am trying to execute multiple spring cloud task jobs within spring cloud data flow container on PCF. These jobs reads the raw file from a http source and then parses it and writes that to mysql db.These jobs are written in plain java and not…
Nilanjan Roy
  • 33
  • 5
0
votes
1 answer

Spring Cloud Dataflow app/stream default properties

I specified in the application.properties: spring.cloud.config.uri=http://configserver:8888 but when I deploy a stream from the dashboard I can see in the logs Fetching config from server at: http://localhost:8888 which means that it still tries…
Adrian Ber
  • 20,474
  • 12
  • 67
  • 117
0
votes
1 answer

contentType getting prefixed to data written from HDFS sink

I am using HDFS sink and writing to HDFS. But the payload I write to HDFS is prefixed with ?contentType "text/plain" though this in not in the payload. Please let me know why this is getting prefixed and how to remove it. stream create…
prash
  • 5
  • 1
0
votes
1 answer

How to send messages to two different output channels in SCDF?

I have a source that sends messages over the default output channel to a processor in the stream. Now I want to also send failure messages over a different channel. I figured I should create a bindable interface that extends from Source and adds the…
Remon Sinnema
  • 51
  • 1
  • 10
0
votes
2 answers

How to handle global resources in spring cloud dataflow?

I'm learning the concepts of spring cloud dataflow and wondering what is the common way of storing global resources. For example, when I have a stream with a pmml-processor and I would like to retrain the underlying pmml-model periodically via a…
Fabian Braun
  • 3,612
  • 1
  • 27
  • 44
0
votes
1 answer

Kafka source and HDFS sink in Spring cloud Data flow

I am using Kafka as the source and I want to write the messages on Kafka to HDFS using HDFS sink.But I see the file getting created on the HDFS but the message on Kafka is not written to the HDFS file.Please find below the Stream DSL. stream create…
prash
  • 5
  • 1
0
votes
1 answer

Spring Cloud Data Flow server unregister all apps on restart

I am able to setup spring cloud data flow server on my local machine. I have setup basic kafka based SCDF server setup by following the link as below: https://cloud.spring.io/spring-cloud-dataflow/#quick-start Now when I bulk import out-of-the-box…
alihur
  • 5
  • 3
0
votes
1 answer

Spring XD: stream and source are inbound/outbound channel adapters?

The source component in Spring XD is designed always like a inbound channel adapter by Spring Integration? Same question about sink component: is always designes like outbound channel adapter? In the doc here about the sink and here about the source…
strano
  • 171
  • 12
1 2 3
…
84 85