0

I saw in the tests spring cloud dataflow used to store the SpringDefinition - HashMap, is it possible to override the configuration of DateFlowServerConfiguration for storing streams and Tasks in an InMemory, for example in the same HashMap, if so, how?

Roman
  • 61
  • 1
  • 6

1 Answers1

1

I don't think it would be a trivial change. The server needs a backend to store it's metadata. By default it actually uses H2 in memory, and it relies on Spring Data JPA abstraction to give users the chance to select their RDBMS.

Storing on a different storage engine, would require not only replacing all the *Repository definitions on several configuration modules, but we do as well some pre population of data. It would become a bit hard to maintain this over time.

Is there a reason why a traditional RDBMS is not suitable here? or if you want in-memory just go with the ephemeral approach of H2?

Vinicius Carvalho
  • 3,994
  • 4
  • 23
  • 29
  • We use the microservice architecture in the project, the microservice uses as data storage - Apache Ignite, and we combined Ignite into a cluster for replicas of the same microservice, I would like to store Streams and Tasks also in Ignite. Because, as far as I know, the built-in H2 doesn't support clustering. – Roman May 04 '17 at 09:46
  • Just because you have a highly distributed architecture, it does not means all your modules will need that. The storage system of SCDF is just used for metadata. Any reliable RDBMS can securely store it for you. You can use a replicated DB such as mysql galera if you are worried about resiliency, but it's not needed to have multiple versions of the data stored at the same time. This is different than the deployment of the streams. – Vinicius Carvalho May 08 '17 at 13:32