0

We need to use the HDFS sink to store some data in HDFS. When creating the stream definition, I can use the "directory" property to specify the hdfs directory where we want to save the file. However, in our use case we want the directory to be dynamic based on the timestamp. I was thinking if it's possible to maybe use some SpEL in the directory property of the HDFS sink to maybe extract the timestamp from the header?

Thanks!!

Rodrigo
  • 1
  • I found this ticket which is related to this funcionality: https://jira.spring.io/browse/XD-274 It would be nice to have a SpeLNamingStrategy which can actually evaluate a SpEL and return the appropiate name – Rodrigo Apr 22 '14 at 22:39

1 Answers1

1

Rodrigo already knows this, but for anyone else checking out this thread, see the explanation of the new functionality by searching for "partitionPath" under this section of the reference doc: https://github.com/spring-projects/spring-xd/wiki/Sinks#hadoop-hdfs

mfisher
  • 506
  • 2
  • 3