0

I am trying to implement simple SFTP channel adapter to accept files with any file name(to allow duplicate file names). For the first poll the file is transferred from SFTP server directory to the local directory and the service activator which is subscribed to this channel is able to receive the message but for the second time if i keep the file with same name in SFTP server, the file is transferred but the service activator was unable to get the message. I tried to implement local filter but it went into infinite loop (polling the file and creating the message).

configuration

<int:channel id="inboundMGetRecursive">
    <int:queue/>
</int:channel>

<int-sftp:inbound-channel-adapter id="sftpInboundAdapter"
                                  auto-startup="true"
                                  channel="channel1"
                                  session-factory="sftpSessionFactory"
                                  local-directory="c:/tmp/"
                                  remote-directory="${sftp.file.remote.inbound.dir}"
                                  auto-create-local-directory="false"
                                  filename-pattern="*.txt"
                                  local-filter="acceptAll">
    <int:poller fixed-delay="60000" error-channel="sftpExceptionsChannel"
                max-messages-per-poll="-1"/>
<bean id="acceptAll" class="org.springframework.integration.file.filters.AcceptAllFileListFilter"/
<int:service-activator id="serviceActivator" input-channel="channel1" ref="fileMoverHandler"
                       method="method1">
    <int:poller fixed-rate="3000" max-messages-per-poll="-1"/>
</int:service-activator>

Infinite loop logs

2016-12-14 13:25:39,632 [task-scheduler-1] [handlers.FileFilter] INFO  - Filter Activated
2016-12-14 13:25:39,635 [task-scheduler-1] [handlers.FileFilter] INFO  - file name : config.txt
2016-12-14 13:25:39,636 [task-scheduler-1] [file.FileReadingMessageSource] DEBUG - Added to queue: [C:\tmp\csvfiles\staging\config.txt]
2016-12-14 13:25:39,636 [task-scheduler-1] [file.FileReadingMessageSource] INFO  - Created message: [GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=4a062556-4181-2806-d2ee-c0aa3221aec3, timestamp=1481743539636}]]
2016-12-14 13:25:39,636 [task-scheduler-1] [endpoint.SourcePollingChannelAdapter] DEBUG - Poll resulted in Message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=4a062556-4181-2806-d2ee-c0aa3221aec3, timestamp=1481743539636}]
2016-12-14 13:25:39,636 [task-scheduler-1] [channel.QueueChannel] DEBUG - preSend on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=4a062556-4181-2806-d2ee-c0aa3221aec3, timestamp=1481743539636}]
2016-12-14 13:25:39,636 [task-scheduler-1] [channel.QueueChannel] DEBUG - postSend (sent=true) on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=4a062556-4181-2806-d2ee-c0aa3221aec3, timestamp=1481743539636}]
2016-12-14 13:25:39,637 [task-scheduler-1] [handlers.FileFilter] INFO  - Filter Activated
2016-12-14 13:25:39,637 [task-scheduler-1] [handlers.FileFilter] INFO  - file name : config.txt
2016-12-14 13:25:39,637 [task-scheduler-1] [file.FileReadingMessageSource] DEBUG - Added to queue: [C:\tmp\csvfiles\staging\config.txt]
2016-12-14 13:25:39,638 [task-scheduler-1] [file.FileReadingMessageSource] INFO  - Created message: [GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=a2df4814-f759-0440-8ce2-c8d969a9315d, timestamp=1481743539638}]]
2016-12-14 13:25:39,638 [task-scheduler-1] [endpoint.SourcePollingChannelAdapter] DEBUG - Poll resulted in Message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=a2df4814-f759-0440-8ce2-c8d969a9315d, timestamp=1481743539638}]
2016-12-14 13:25:39,638 [task-scheduler-1] [channel.QueueChannel] DEBUG - preSend on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=a2df4814-f759-0440-8ce2-c8d969a9315d, timestamp=1481743539638}]
2016-12-14 13:25:39,639 [task-scheduler-1] [channel.QueueChannel] DEBUG - postSend (sent=true) on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=a2df4814-f759-0440-8ce2-c8d969a9315d, timestamp=1481743539638}]
2016-12-14 13:25:39,639 [task-scheduler-1] [handlers.FileFilter] INFO  - Filter Activated
2016-12-14 13:25:39,639 [task-scheduler-1] [handlers.FileFilter] INFO  - file name : config.txt
2016-12-14 13:25:39,640 [task-scheduler-1] [file.FileReadingMessageSource] DEBUG - Added to queue: [C:\tmp\csvfiles\staging\config.txt]
2016-12-14 13:25:39,640 [task-scheduler-1] [file.FileReadingMessageSource] INFO  - Created message: [GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=fd1ace38-6550-82ea-89c5-2a6229463167, timestamp=1481743539640}]]
2016-12-14 13:25:39,640 [task-scheduler-1] [endpoint.SourcePollingChannelAdapter] DEBUG - Poll resulted in Message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=fd1ace38-6550-82ea-89c5-2a6229463167, timestamp=1481743539640}]
2016-12-14 13:25:39,640 [task-scheduler-1] [channel.QueueChannel] DEBUG - preSend on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=fd1ace38-6550-82ea-89c5-2a6229463167, timestamp=1481743539640}]
2016-12-14 13:25:39,640 [task-scheduler-1] [channel.QueueChannel] DEBUG - postSend (sent=true) on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=fd1ace38-6550-82ea-89c5-2a6229463167, timestamp=1481743539640}]

Please help me on how can i use filter to accept all files.

Abhilash
  • 458
  • 2
  • 7

1 Answers1

1

When using the accept all local filter, you need to remove/rename the file from the local disk when processing is complete, before the next poll.

Unless you also remove the remote file, you should also use an FtpPersistentAcceptOnceFileListFilter int the remote filter so the file is not re-fetched until its modified timestamp changes.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • Thanks Gary for reply. But it is polling many times within a second although i have 1 minute as polling fixed-delay. Please correct me if I am missing something. – Abhilash Dec 14 '16 at 20:06
  • Don't use a queue channel; remove `` from the channel and remove the poller from the service activator. It will now run on the adapter's poller thread. The file must be removed before the adapter poller's thread is returned to the poller. – Gary Russell Dec 14 '16 at 20:09
  • I have removed queue from channel and also poller from service activator but still for the second time inbound channel adapter polls multiple times in 1 sec. This is happening when i keep local-filter only. – Abhilash Dec 14 '16 at 21:57
  • With `max-messages-per-poll="-1"` we will keep returning files until none pass the filter. Then the poller sleeps. If you only want one file per poll use `="1"`. – Gary Russell Dec 14 '16 at 22:00