2

In Spring Cloud Data Flow (1.3.0) I can create and destroy streams, but how can I edit existing streams? E.g. if I want to temporarily route something to a log, and then later remove the logging, do I have to destroy and then re-create the streams? Or is there a recommended workflow for editing existing streams?

Jerry
  • 1,127
  • 2
  • 17
  • 35

2 Answers2

3

You can't edit streams, the nature of the topology makes it very hard for to guarantee some constraints, for instance what should you do with in flight messages?

However, we have a new project Spring Cloud Skipper That makes it easy to version and update your streams, take a look at the docs, and try some of the samples, you will see that creating new versions of streams is a much easier task now.

Vinicius Carvalho
  • 3,994
  • 4
  • 23
  • 29
  • So instead of editing an existing stream, the current practice is to create a whole new stream from scratch with the new features and then undeploy the old stream and deploy the new stream? This seems cludgy and repetitive. Are there any plans to improve this? – James Wynn May 09 '18 at 16:03
1

To add to what Vinny said, in 1.3, we have added the support to "update" a stream. Following are the options that are available when using this feature.

  • upgrade/rollback individual app-versions
  • change deployment properties of apps
  • change application properties of apps

All this can be done without undeploying or destroying the stream. Skipper backs the core of the work and it also keeps track of the update-history, so at any given point in time, you can forward or rewind as needed. See docs for more details.

We will get to editing the streaming-topology shapes (e.g., add or remove steps/apps) in the upcoming releases.

EDIT:

If you want to temporarily route the copy of the upstream events to a log-sink or other destination, you can use the TAP feature in SCDF. This recent screencast shows it in action.

Sabby Anandan
  • 5,636
  • 2
  • 12
  • 21