2

I have a spring integration app deployed on two nodes in a cluster. they poll for incoming files on single directory (inbound file adapter) . If a file is picked up by a node and processed,sometimes the other node throws the below exceptions ,by trying to pick up the same file.

ERROR org.springframework.integration.handler.LoggingHandler - org.springframework.integration.MessageHandlingException: failed to write Message payload to file ..... Caused by: java.io.FileNotFoundException: /somedir/dir/file.txt (No such file or directory)

I know polling /clustering etc has been discussed a lot everywhere . did find an accurate explanation for this anywhere .I am trying to clean this up and keep the logs more cleaner.thanks in advance. Is there any simple config in the inbound file adapter to prevent this.

Zak
  • 111
  • 3
  • 11

1 Answers1

2

One solutions is to use a single source polling the directory and distribute the work to competing consumers using some intermediate transport, such as RabbitMQ, JMS etc.

Another is to use a FileSystemPersistentAcceptOnceFileListFilter which uses a ConcurrentMetadataStore so only one of the instances will see each file.

This needs some shared state - the framework provides a RedisMetadataStore as a built-in option.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179