moving a file to different location while it is still been accessed

Question

I developed a java program which is writing data into a file(temporary file) using asynchronousFileChannel.
Now each time it writes a entry to the file a second method will be called which will check for its size and if its size exceeds a pre thresold size say 10000 bytes,all the file contents should be moved to a different file(permanent Datarecord).
What should i use so that no data should be lost while moving data from temp file to permanent file while the temporary file is still been accessed by different threads.

use another file for writing (ex: tempFileA, tempFileB). move the first file while continuing to write in the second. — Lescai Ionel, Sep 18 '13 at 06:29
Most OS will lock the file while it's open for writing and there's no way you can move it to a different location before the lock is released. After you FINISHED writing the file you can check the size and move it accordingly. — Nir Alfasi, Sep 18 '13 at 06:40
i can handle the lock by checking the bytes in the same method i am using the asynchronousFileChannel ,but not able to a find appropriate way to move the file,while deleting the content of the temporary file which is already moved to the permanent file. — Charles Stevens, Sep 18 '13 at 07:29

score 2 · Answer 1 · answered Sep 18 '13 at 06:32

2

10,000 bytes is not very much, it is just <10kb. So I think you can buffer all this data in queue and when size exceeds you can clear(delete-create) temporary file and flush queue data to permanent storage file.

answered Sep 18 '13 at 06:32

cdkrot

279
1
2
8

score 2 · Answer 2 · edited May 23 '17 at 11:56

2

Since you are using AsynchronousFileChannel, there is always a problem that when one of your methods is trying to move the file, another thread might be trying to read/write to it.

You need to do the move by other means.

If what you are doing is logging, which I think you might be doing - use a FileHandler to roll the file. See an example here - http://kodejava.org/how-do-i-create-a-rolling-log-files/

You can also see some discussion about rolling files here - Rolling file implementation

Another one on stackoverflow - Rolling logs by both size and time

And another one - How to write statistical data into rolling files in java

Hope this helps. Good Luck.

edited May 23 '17 at 11:56

Community

1
1

answered Sep 18 '13 at 07:11

Indu Devanath

2,068
1
16
17

I am not doing logging just simply writing a file using asynchronousFileChannel with its lock() method while writing,not able to find a method to move it if the size exceeds the threshold to a permanent directory – Charles Stevens Sep 18 '13 at 07:21
If you are using AsynchronousFileChannel, at any given time, multiple access will be happening and you will not get a proper lock() on it. Did you try the other links. There is some sample code you could use to try to do it. – Indu Devanath Sep 18 '13 at 10:36
If you are using AsynchronousFileChannel, at any given time, multiple access will be happening and you will not get a proper lock() on it. You might have to tryLock() first to check if you can even acquire a lock on it. I feel rolling a file after the file gets to the size is a better option. Check the links and see if any of those solutions work for you. Post the code you are having trouble with. May be we might get a better idea and can provide you with a solution. – Indu Devanath Sep 18 '13 at 10:44
could u suggest some links where i can get the idea for rolling a file. – Charles Stevens Sep 18 '13 at 11:49

user207421 · Answer 3 · 2013-09-18T08:15:10.570

1

Just close the file, rename it, and open a new one. You're overthinking this.

edited Sep 18 '13 at 08:15

answered Sep 18 '13 at 06:48

user207421

305,947
44
307
483

score 1 · Accepted Answer · answered Sep 19 '13 at 07:22

Here's the code I wrote for you...

public class RollMyFile {

    private long FILE_MAX_SIZE;
    private String fileName;

    /**
     * Constructor to take in the initial file name and the maxFileSize
     * @param fileNameToStartWith
     */
    public RollMyFile(String fileNameToStartWith, long maxFileSize) {
        this.fileName = fileNameToStartWith;
        this.FILE_MAX_SIZE = maxFileSize;
    }

    /**
     * Synchronized to roll over to a new file
     * 
     * @param fileChannel
     * @return
     * @throws IOException
     */
    public synchronized AsynchronousFileChannel rollMeIfNeeded(AsynchronousFileChannel fileChannel) throws IOException {
        if(fileChannel.size()>FILE_MAX_SIZE) {
            this.fileName = getNewRolledFileName(this.fileName);
            File file = new File(this.fileName);
            file.createNewFile();
            fileChannel = getAsyncChannel(this.fileName);
        }
        return fileChannel;
    }

    /**
     * Change this to create a new name for the roll over file
     * @param currentFileName
     * @return
     */
    public String getNewRolledFileName(String currentFileName) {
        if (currentFileName.contains(".")) {
            currentFileName = currentFileName.substring(0,
                    currentFileName.lastIndexOf('.'));
        }
        return currentFileName+ "." + Calendar.getInstance().getTimeInMillis();
    }

    /**
     * This is where you request to write a whole bunch of stuff that
     * you said you want to store
     * 
     * @param stuffToWrite
     * @throws IOException
     */
    public void write(StringBuffer stuffToWrite) throws IOException {
        AsynchronousFileChannel fileChannel = getAsyncChannel(this.fileName);
        fileChannel = rollMeIfNeeded(fileChannel);
        ByteBuffer byteBuffer = ByteBuffer.wrap(stuffToWrite.toString().getBytes());
        fileChannel.write(byteBuffer, fileChannel.size());
    }

    /**
     * Change this to how ever you 'open' the AsynchronousFileChannel
     * 
     * @param givenFileName
     * @return
     * @throws IOException
     */
    private AsynchronousFileChannel getAsyncChannel(String givenFileName) throws IOException {
        return AsynchronousFileChannel.open(Paths.get(givenFileName), StandardOpenOption.WRITE);
    }

}

And I used it like below

public class Tester {

    public static void main(String[] args) {

        RollMyFile rmf = new RollMyFile("my-current-file", 1000);
        try {
            for(int i=1; i<1000; i++) {
                rmf.write(new StringBuffer(" lots of important stuff to store... "));
                System.out.println(i);
            }
            System.out.println("end");
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

    }

}

moving a file to different location while it is still been accessed

4 Answers4