How can I create a route with multiple connected ftp calls using Camel and Java DSL?

Question

I have this synchronous pipeline that need to be executed from time to time (lets say every 30 minutes):

1. Connect to a ftp;
2. Read a .json file (single file) from folder A;
3. Unmarshall the content of the file (Class A) and add it to the route context;
4. Read all the .fixedlenght files (multiple files) from folder B (preMove: processingFolder, move: doneFolder, moveFailed: errorFolder);
5. Unmarshall the content of the files (Class B) and do some logic;
6. Read all the .xml files (multiple files) from folder C (preMove: processingFolder, move: doneFolder, moveFailed: errorFolder);
7. Unmarshall the content of the files (Class C) and do some logic;
8. End the route.

It is a single pipeline created with Java DSL. If a error happen, the process stop. I'm really struggling with Camel to create this. It is possible or I will need to handle this manually? I created some demos, but none of them are properly working.

Any help will be appreciated.

Answer 1

The way you describe your message pipeline it seems beneficial to have 3 separate routes each handling a different folder in your FTP server. You can have a timer that triggers all 3 every 30 minutes of so. The FTP component derives from Camel's File Component and there are a lot a useful parameters that would help with your routing logic here.

For each of your 3 routes you would have something like this:

from("ftp://foo@myserver?include=*.xml&preMove=processingFolder&move=doneFolder&moveFailed=errorFolder")
    .unmarshal()
    ...

You can find more info about filtering files by their extensions here

Answer 2

I would approach this in the following manner:

1. All the interfaces to the FTP where you read the files are separate routes. Their job is only to pick up the file. They don't deal with parsing or transformation.

2. Then create separate routes for actually receiving the data, parsing and transformation.

3. Finally the delivery routes which take the data and deliver to your end destination.

This way you can customise the error handling, easier to find out what went wrong were, makes it easier to change one part without affecting everything and you can reuse the routes in several different parts.