How can I rename a file placed in remote location while it is getting streamed

Question

I want to add some temporary suffix to file while I am streaming the file from remote directory.

I am streaming the file from remote directory using spring integration dsl and I want to make sure that one file is getting read by single application at a time. So i am thinking to adding some temporary prefix to file while it is getting streamed. I am using outbound gateway to fetch the data.

Any pointers will be very helpful. Currently I am renaming the file before reading and after reading, I really don`t want to do that.

Lucifer Morningstar · Answer 1 · 2019-06-20T03:17:29.437

consider using file locking, instead of renaming. Here is the relevant part from the 13.2 Reading Files spring-integration documentation:

When multiple processes are reading from the same directory it can be desirable to lock files to prevent them from being picked up concurrently. To do this you can use a FileLocker. There is a java.nio based implementation available out of the box, but it is also possible to implement your own locking scheme. The nio locker can be injected as follows:

<int-file:inbound-channel-adapter id="filesIn"
    directory="file:${input.directory}" prevent-duplicates="true">
    <int-file:nio-locker/>
</int-file:inbound-channel-adapter>

A custom locker you can configure like this:

<int-file:inbound-channel-adapter id="filesIn"
    directory="file:${input.directory}" prevent-duplicates="true">
    <int-file:locker ref="customLocker"/>
</int-file:inbound-channel-adapter>

[Note] When a file inbound adapter is configured with a locker, it will take the responsibility to acquire a lock before the file is allowed to be received. It will not assume the responsibility to unlock the file. If you have processed the file and keeping the locks hanging around you have a memory leak. If this is a problem in your case you should call FileLocker.unlock(File file) yourself at the appropriate time.

Please see the docs for Interface FileLocker and Class NioFileLocker for more info.

I am using Sftp.outboundGateway(sftpSessionFactory(), GET, "payload.remoteDirectory + payload.filename").options(STREAM).temporaryFileSuffix("_reading"). How can I apply locking in this. — Springy Developer, Jun 20 '19 at 03:01
Please see the docs for [Interface FileLocker][2] and [Class NioFileLocker][3] for more info. (see links at the bottom of the above post) — Lucifer Morningstar, Jun 20 '19 at 03:18

score 0 · Answer 2 · answered Jun 20 '19 at 02:27

I would use the Apache Commons FileUtils. https://commons.apache.org/proper/commons-io/javadocs/api-2.5/org/apache/commons/io/FileUtils.html#moveFile

Typically what I do is write the file during initial transfer to a temporary working directory. After the file is completely transferred I do a checksum to guarantee the file is correct. At this point I move the file to the final directory being used by the other applications logic. As long as the working directory and the final directory are on the same filesystem the move will be atomic. This will guarantee no race conditions between the different parts of the application using the file.

How can I rename a file placed in remote location while it is getting streamed

2 Answers2