Opening a file output stream on a file that is already locked overwrites it

Question

I came across this scenario and did not understand why it is happening. Can someone please help me understand the behaviour of nio file lock.

I opened a file using FileOutputStream and after acquiring an exclusive lock using nio FileLock I wrote some data into the file. Did not release the lock. Opened another FileOutputStream on the same file with an intention to acquire a lock and do a write operation and expect this to fail.But opening the second fileoutputstream overwrote the already locked file which had data written into it even before I try to get second lock. Is this expected? My understanding was acquiring an exclusive lock would prevent any changes on the locked file. How can I prevent overwriting my file when trying to get another lock ? (as if another process tries to get a lock on the same file on a different vm ? )

Sample program I tried:

        File fileToWrite = new File("C:\\temp\\myfile.txt");
        FileOutputStream fos1 = new FileOutputStream(fileToWrite);
        FileOutputStream fos2 =null;
        FileLock lock1,lock2 =null;
        lock1=fos1.getChannel().tryLock();
        if(lock1!=null){
            //wrote date to myfile.txt after acquiring lock
            fos1.write(data.getBytes());
            //opened myfile.txt again and this replaced the file
            fos2 = new FileOutputStream(fileToWrite);
            //got an overlappingfilelock exception here
            lock2=fos2.getChannel().tryLock();
            fos2.write(newdata.getBytes());
            }

        lock1.release();
        fos1.close();
        if(lock2!=null)
            lock2.release();
        fos2.close();

Also tried splitting the above into two programs. Executed 1st and started second when 1st is waiting. File which is locked by program1 got overwritten by program2. Sample below:

Program1:

    File fileToWrite = new File("C:\\temp\\myfile.txt");
    FileOutputStream fos1 = new FileOutputStream(fileToWrite);
    FileLock lock1 =null;
    lock1=fos1.getChannel().tryLock();
    if(lock1!=null){
        //wrote date to myfile.txt after acquiring lock
        fos1.write(data.getBytes());
        System.out.println("wrote data and waiting");
        //start other program while sleep
        Thread.sleep(10000);
        System.out.println("finished wait");
        }

    lock1.release();
    fos1.close();

Program2:

   File fileToWrite = new File("C:\\temp\\myfile.txt");
    System.out.println("opening 2nd out stream");
    //this overwrote the file
    FileOutputStream fos2 = new FileOutputStream(fileToWrite);
    FileLock lock2 =null;
    lock2=fos2.getChannel().tryLock();
    //lock is null here
    System.out.println("lock2="+lock2);
    if(lock2!=null){
        //wrote date to myfile.txt after acquiring lock
        System.out.println("writing  NEW data");
        fos2.write(newdata.getBytes());
        }

    if(lock2!=null)
        lock2.release();
    fos2.close();

Thanks

Answer 1

When you acquire a FileLock, you acquire it for the entire JVM. That’s why creating more FileOutputStreams and overwriting the same file within the same JVM will never been prevented by a FileLock— the JVM owns the lock. Thus, the OverlappingFileLockException is not meant to tell you that the lock isn’t available (that would be signaled by tryLock via returning null), it’s meant to tell you that there is a programming error: an attempt to acquire a lock that you already own.

When trying to access the same file from a different JVM, you stumble across the fact that the locking isn’t necessarily preventing other processes from writing into the locked region, it just prevents them from locking that region. And since you are using the constructor which truncates existing files, that might happen before your attempt of acquiring the lock.

One solution is use new FileOutputStream(fileToWrite, true) to avoid truncating the file. This works regardless of whether you open the file within the same JVM or a different process.

However, maybe you don’t want to append to the file. I guess you want to overwrite in the case you successfully acquired the lock. In this case, the constructors of FileOutputStream don’t help you as they force you to decide for either, truncating or appending.

The solution is to abandon the old API and open the FileChannel directly (requires at least Java 7). Then you have plenty of standard open options where truncating and appending are distinct. Omitting both allows overwriting without eagerly truncating the file:

try(FileChannel fch=FileChannel.open(fileToWrite.toPath(),
                                     StandardOpenOption.CREATE, StandardOpenOption.WRITE)){
  try(FileLock lock=fch.tryLock()) {
    if(lock!=null) {
      // you can directly write into the channel
      // but in case you really need an OutputStream:
      OutputStream fos=Channels.newOutputStream(fch);
      fos.write(testData.getBytes());
      // you may explicitly truncate the file to the actually written content:
      fch.truncate(fch.position());
      System.out.println("waiting while holding lock...");
      LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(5));
    }
    else System.out.println("couldn't acquire lock");
  }
}

Since it requires Java 7 anyway you can use automatic resource management for cleaning up. Note that this code uses CREATE which implies the already familiar behavior of creating the file if it doesn’t exists, in contrast to CREATE_NEW which would require that the file doesn’t exist.

Due to the specified options, the open operation may create the file but not truncate it. All subsequent operations are only performed when acquiring the lock succeeded.

Answer 2

File locks only are only specified to work against other file locks.

From the Javadoc:

Whether or not a lock actually prevents another program from accessing the content of the locked region is system-dependent and therefore unspecified. The native file-locking facilities of some systems are merely advisory, meaning that programs must cooperatively observe a known locking protocol in order to guarantee data integrity. On other systems native file locks are mandatory, meaning that if one program locks a region of a file then other programs are actually prevented from accessing that region in a way that would violate the lock. On yet other systems, whether native file locks are advisory or mandatory is configurable on a per-file basis. To ensure consistent and correct behavior across platforms, it is strongly recommended that the locks provided by this API be used as if they were advisory locks.