0

I'm newbie in apache camel. I configured a route from camel ftp to JMS topic. I noticed that the processed files, they are either deleted, moved either processed again and again. In my work, I have a lot of corrupt files that are loaded into HDFS, so I need to reentered the same files into FTP, I don't need to send again the full file, but this time extract only the new lines (appended lines).

Thanks,

Chayma Sakouhi
  • 79
  • 2
  • 10
  • Please add more information to your question as currently, it's completely unclear what was is expected behaviour, what was already tried etc. – Vyacheslav Enis May 15 '17 at 10:54
  • I want to send files from ftp server to hdfs using apache camel + flume. So I configure camel route from ftp to JMS topic, then, the flume agent consume files from jms source to hdfs sink. However, sometimes my ftp server receives an incomplete files. Once the files are processed and sent to hdfs. I need to reenter the same files. The possible options in ftp component is to delete processed file, to move it or to (Override, append .....) and send it again. In this case, I want to send the new lines added to jms --> flume --> hdfs. – Chayma Sakouhi May 15 '17 at 11:22
  • I have a different case. I work on Big data project. Sometimes the FTP server receives an incomplete files. Then, I need to load them again. I no want to have a duplicate files into hdfs. I just need to load the new lines. I look for an option that scan the new lines and send them. – Chayma Sakouhi May 15 '17 at 11:35

0 Answers0