0

we are currently using SpringXD to ingest the data from the various data sources to the sinks. Since Springxd is scheduled for end of availability, we would like to migrate to Spring Cloud data flow.

  1. We have written several custom modules for SpringXD. How to use the same modules in SCDF. What are the changes required to use these in SCDF.
  2. Our cluster is kerberos configured, does SCDF supports kerberized kafka and hadoop?.
  3. Do we have any issues when using the server in the yarn for runtime.
  4. Please share us the challenges when migrating from XD to SCDF.

Any help appreciated Thanks in advance

nagendra
  • 1,885
  • 3
  • 17
  • 27

1 Answers1

0

You can refer this documentation for the migration guide here

For the kerberos configuration of Kafka Spring Cloud Stream binder, you can refer the documentation here

Ilayaperumal Gopinathan
  • 4,099
  • 1
  • 13
  • 12
  • For hadoop kerberos, you can use the spring hadoop kerberos properties as mentioned here: http://docs.spring.io/spring-hadoop/docs/current/reference/htmlsingle/#literal-spring-hadoop-security-literal-configuration-properties – Ilayaperumal Gopinathan Apr 27 '17 at 11:34
  • Currently we are on springxd and our cluster security changed to kerberos. We are not able to publish the messages to Kafka due to kerberos failure. Is there any workaround to configure kafka with kerberos in springxd – nagendra Apr 27 '17 at 11:44
  • I believe you need to configure those explicitly in the XD module. For instance, for the Kafka sink, you can set the producer configs like `security.protocol` and `sasl.kerberos.service.name` here: https://github.com/spring-projects/spring-xd/blob/master/modules/sink/kafka/config/kafka.xml#L15 – Ilayaperumal Gopinathan Apr 27 '17 at 12:04