0

I am just starting to look at apache camel (Using blueprint routes) and I am already stuck.

I need to process a set of csv files with different formats. I get 5 files with foo_X_20160110.csv were X specifies the type of csv file and the files have a date stamp . These files can be quite large so a 'done' file is written once all files are written. The done file is named foo_trigger_20160110.csv.

I've seen the doneFileName option on file but that only supports a static name (I have a date in the filename) or it expects a done file for each input file.

The files have to be proceeded in a fixed order but it is not guaranteed in which order they are written to the input directory. Hence I need to wait for the done file.

Any idea how this can be done with Camel?

Any suggestions for good Camel books?

Ben
  • 1,922
  • 3
  • 23
  • 37
  • Hi, those were several questions in once. Can you specify in more detail which question you are after? As for books, you should start with Camel in Action which is the main book, then there is Camel Cookbook as well. – Souciance Eqdam Rashti Jan 27 '16 at 22:26
  • Thanks for the book recommendation. The main question was on how I can configure the file URI to use a non-static done file and only process the other files in the directory when that 0-byte done file (with changing name based on date) is present – Ben Jan 28 '16 at 07:39

1 Answers1

2

Here is an example from the documentation http://camel.apache.org/file2.html

from("file:C:/temp/input.txt?doneFileName=done");

As you can see the doneFileName has a static value "done". But you can use standard java to write dynamic names i.e. for current dateformat or anything else and just use string operation to construct the URI. Hope that helps.

Update:

By the way, as mentioned in the documentation there is the option of dynamic placeholders for the doneFileName.

However its more common to have one done file per target file. This means there is a 1:1 correlation. To do this you must use dynamic placeholders in the doneFileName option. Currently Camel supports the following two dynamic tokens: file:name and file:name.noext which must be enclosed in ${ }. The consumer only supports the static part of the done file name as either prefix or suffix (not both).

from("file:bar?doneFileName=${file:name}.done");

You can also use a prefix for the done file, such as:

from("file:bar?doneFileName=ready-${file:name}");
Souciance Eqdam Rashti
  • 3,143
  • 3
  • 15
  • 31
  • Thanks, can you expand on how you would invoke Java to create the dynamic name from the file URI? Sorry still early days. The Camel In Action book is arriving tomorrow – Ben Jan 28 '16 at 11:11
  • I guess one option would be to use a custom processStrategy using a GenericFileProcessStrategy? – Ben Jan 28 '16 at 11:27
  • I don't think you need that. With standard java I mean, if the done file has the name "currentdate".txt where currentdate is the today's date. Then you simply find that using standard java and concatenate that to create the URI. That should be one way to get dynamic done file names. – Souciance Eqdam Rashti Jan 28 '16 at 11:29
  • If I set up the route with Java DSL yes I can easlily do that for the first day, but that app might be running for multiple days and the route is only set up once. So how would I change that? And how would I get the dynamic file name when using blueprint XML? – Ben Jan 28 '16 at 17:36
  • I am not sure what you mean by the first date. In java you can get the currentdate which means the date of today, it will give you date when you run it tomorrow, day after etc. It is a method. String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(Calendar.getInstance().getTime()); – Souciance Eqdam Rashti Jan 28 '16 at 18:45
  • For blueprint, you have to probably create a processor, which gives you access to writing java code, do the same thing there, and populate an exchange header, and then in the blueprint dsl combine the two. In all honestly, I would advise to use blueprint only for loading the routes. The rest, i.e routing logic and everything else, do that in the java dsl. Much easier. – Souciance Eqdam Rashti Jan 28 '16 at 18:47