Spring integration - SFTP rename or move file in remote server after copying

Question

I am trying to move or rename remote file instead of deleting the remote file after download, I found that it can be done by outbound gateway move command, but could not find proper way to do it.

Please help in rename the file after download.

@Bean
@Order(Ordered.HIGHEST_PRECEDENCE)
public SessionFactory<LsEntry> sftpSessionFactory(
        final DataloadServiceProperties DataloadServiceProperties) {
    DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
    factory.setHost(DataloadServiceProperties.getSftpHost());
    factory.setPort(DataloadServiceProperties.getSftpPort());
    factory.setUser(DataloadServiceProperties.getSftpUser());
    if (DataloadServiceProperties.getSftpPrivateKey() != null) {
        factory.setPrivateKey(DataloadServiceProperties.getSftpPrivateKey());
        factory.setPrivateKeyPassphrase(
                DataloadServiceProperties.getSftpPrivateKeyPassphrase());
    }
    else {
        factory.setPassword(DataloadServiceProperties.getSftpPasword());
    }
    factory.setAllowUnknownKeys(true);
    return new CachingSessionFactory<LsEntry>(factory);
}

@Bean
@Order(Ordered.HIGHEST_PRECEDENCE - 1)
public SftpInboundFileSynchronizer sftpInboundFileSynchronizer(
        final SessionFactory<LsEntry> sftpSessionFactory,
        final DataloadServiceProperties DataloadServiceProperties) {
    SftpInboundFileSynchronizer fileSynchronizer =
            new SftpInboundFileSynchronizer(sftpSessionFactory);
    fileSynchronizer.setDeleteRemoteFiles(false);
    fileSynchronizer.setRemoteDirectory(
            DataloadServiceProperties.getSftpRemoteDirectoryDownload());
    fileSynchronizer.setFilter(new SftpSimplePatternFileListFilter(
            DataloadServiceProperties.getSftpRemoteDirectoryDownloadFilter()));
    return fileSynchronizer;
}

Inbound channel for looking file in SFTP server

@Bean
@Order(Ordered.HIGHEST_PRECEDENCE - 2)
@InboundChannelAdapter(
        channel = "fromSftpChannel",
        poller = @Poller(
                cron = "${sftp.poller.cron}"))
public MessageSource<File> sftpMessageSource(
        final SftpInboundFileSynchronizer sftpInboundFileSynchronizer,
        final DataloadServiceProperties DataloadServiceProperties) {
    SftpInboundFileSynchronizingMessageSource source =
            new SftpInboundFileSynchronizingMessageSource(sftpInboundFileSynchronizer);
    source.setLocalDirectory(
            new File(DataloadServiceProperties.getSftpLocalDirectoryDownload()));
    source.setAutoCreateLocalDirectory(true);
    source.setLocalFilter(new AcceptOnceFileListFilter<File>());
    return source;
}

Processing the file after downloaded to local folder

@Bean
@Inject
@ServiceActivator(
        inputChannel = "fromSftpChannel")
public MessageHandler resultFileHandler() {
    return new MessageHandler() {
        @Override
        public void handleMessage(final Message<?> message) throws MessagingException {
            String payload = String.valueOf(message.getPayload());
            if (!StringUtils.isEmpty(payload) && payload.endsWith("gz")) {
                LOGGER.info("toRequest : {}", message.getPayload());
            }
        }
    };
}

Thank you Artem Bilan, I have added below code for moving the file to uat folder after download. Its working as expected now.

private static final SpelExpressionParser PARSER = new SpelExpressionParser();

@Bean(name="fromSftpChannel")
 public MessageChannel fromSftpChannel() {
     return new PublishSubscribeChannel();
 }

 @Bean
@Inject
@ServiceActivator(inputChannel = "fromSftpChannel")
@Order(Ordered.LOWEST_PRECEDENCE)
public MessageHandler moveFile() {
    SftpOutboundGateway sftpOutboundGateway = new  SftpOutboundGateway(sftpSessionFactory(), Command.MV.getCommand(), "'/test/'.concat(" + PARSER.parseExpression("payload.getName()").getExpressionString() + ")");
    sftpOutboundGateway.setRenameExpressionString("'/test/uat/'.concat(" + PARSER.parseExpression("payload.getName()").getExpressionString() + ")");
    sftpOutboundGateway.setRequiresReply(false);
    sftpOutboundGateway.setOutputChannelName("nullChannel");
    sftpOutboundGateway.setOrder(Ordered.LOWEST_PRECEDENCE);
    sftpOutboundGateway.setAsync(true);
    return sftpOutboundGateway;
}

score 3 · Accepted Answer · answered Oct 03 '18 at 16:16

3

You need to make that fromSftpChannel as a PublishSubscribeChannel and have a second subscriber with the SftpOutboundGateway. That one you really configure for the Command.MV and that's it! Don't forget to configure setRenameExpression() to specify a remote path for moving!

answered Oct 03 '18 at 16:16

Artem Bilan

113,505
11
91
118

Thank you, I think this is the way to change channel to publish subscribe channel, but did not know how to set rename expression.@Bean(name="fromSftpChannel") public MessageChannel fromSftpChannel() { return new PublishSubscribeChannel(); } – nmanandhan Oct 03 '18 at 16:41
See `MV` command description: https://docs.spring.io/spring-integration/docs/5.0.8.RELEASE/reference/html/sftp.html#sftp-outbound-gateway – Artem Bilan Oct 03 '18 at 16:45
The code is not readable in the comments. Please, consider to edit your question with more info. Also would be to share a result of the execution, but don't wait for such a question from me. – Artem Bilan Oct 03 '18 at 16:56
i have done the recommended changes for moving the file, but no output-channel or replyChannel header available exception thrown, I have added the code changes in question. can you please help on resolve this? – nmanandhan Oct 05 '18 at 10:45
Since `SftpOutboundGateway` is really a request-reply component and there is no *send-only* mode, you need to add `sftpOutboundGateway.setOutputChannelName("nullChannel");` to stop processing afterwards. – Artem Bilan Oct 05 '18 at 13:38
thanks again, this resolved the exception. poller running every 5 secs to look for new file in sftp server. first file processed and moved to folder properly, however poller stopped picking the file from sftp after first successful run. is it because of null channel configuration? – nmanandhan Oct 05 '18 at 13:53
sure, thanks again for closely following up on this. i just placed the file back in sftp server for processing but it did not pick it. – nmanandhan Oct 05 '18 at 13:57
Correct. Because you have there ` source.setLocalFilter(new AcceptOnceFileListFilter());`. So, the same file is not processed again until after application restart. You should consider to use some persistent `FileListFilter` (`SftpPersistentAcceptOnceFileListFilter `) to keep the state in between application restarts and don't process the same files again: https://docs.spring.io/spring-integration/docs/current/reference/html/sftp.html#sftp-inbound – Artem Bilan Oct 05 '18 at 14:03
yes, its working fine. changed the file name in sftp server and it picked up for processing. Thank you – nmanandhan Oct 05 '18 at 14:06
Great! So, no need in a new SO question then! – Artem Bilan Oct 05 '18 at 14:06

Spring integration - SFTP rename or move file in remote server after copying

1 Answers1

Linked