2

What my system does is periodically read a directory from multiple remote ftp servers, and copy the files into a local server. The method I use is the Integration DSL FTP outbound gateway, with DelegatingSessionFactory to listen to multiple remote servers. Performance is very crucial here. Say, there are like 40 FTP servers being listened to. Me, nor my superior are not in the knowledge how DelegatingSessionFactory performance will go, and whether it will fulfill requirement. My superior suggest I create an Integration flow for every FTP servers so that it will listen to all the servers at the same time and presumable copies those files at the same. I feel this sounds crazy. In fact, I feel it might perform even worse. I also do not know the way DelegatingSessionFactory listens to every servers.

Unfortunately we have not tested how the performance is on multiple FTP servers, nor we can test it if we want to. Is there a known performance on DelegatingSessionFactory? Is making individual Integration flow will do any good, as crazy as it sounds? Even if we want to make many flows, is there a way to dynamically create all the flows? My superior even suggested if there is other library which can do the job better.

If DelegatingSessionFactory is the best method, which I think it is, I need to convince my superior that it is the best, by at least maybe know how the listening queue goes.

Thanks in advance

Azinuddin
  • 29
  • 4

1 Answers1

2

The DelegatingSessionFactory is OK in this case. As long as you supply a ThreadLocal key upfront your FtpOutboundGateway.

I would implement it as a single flow starting with some polling message source with a list of keys you would supply for that DelegatingSessionFactory. Then you split such a list to individual messages and emit them into an ExecutorChannel which would give you a parallelism for subsequent FTP servers fetches. Then you call a DelegatingSessionFactory.setThreadKey(Message<?> message, Object key) in the service activator before going to the FtpOutboundGateway.

There is also a way to create inbound channel adapters for all those FTP servers dynamically and dump all of them to the same message channel for the rest of predefined flow.

I would say both approaches are equal by performance, unless you are really going to add more servers at runtime, then you will suffer a little bit for dynamic beans registrations.

See more info in doc about DelegatingSessionFactory and dynamic flow registrations:

https://docs.spring.io/spring-integration/docs/current/reference/html/ftp.html#ftp-dsf

https://docs.spring.io/spring-integration/docs/current/reference/html/dsl.html#java-dsl-runtime-flows

Artem Bilan
  • 113,505
  • 11
  • 91
  • 118