Loosing data with (GatheringByteChannel) RandomAccessFile.getChannel.write(buffers)

Question

I open raf, get file channel, accumulate some data in buffers then

channel.position(raf.length)
channel.write(buffers)
channel.close
raf.close

I expect that bytes are written into raf at these offsets, which are raf.length + accumulated buffer sizes in the array

0,62,132,195,259,322,392,455,519,589,652,716,779,842,905,968,1031,1093,1155,1225,1287,1350,1414,1477,1541,1611,1674,1737,1801,1863,1927,1989,2059,2123,2193,2256,2319,2382,2452,2516,2586,2648,2711,2774,2837,2900,2962,3025,3089,3152,3216,3286,3348,3412,3482,3544,3614,3684,3754,3818,3882,3952,4022,4092

and ProcMon displays that this is what starts to happen at OS level

"CreateFile","objects.bin","SUCCESS","Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
"QueryNetworkOpenInformationFile","objects.bin","SUCCESS","CreationTime: 6.12.2015 20:11:21, LastAccessTime: 8.12.2015 11:03:49, LastWriteTime: 8.12.2015 11:03:49, ChangeTime: 8.12.2015 11:03:49, AllocationSize: 1.01.1601 2:00:00, EndOfFile: 1.01.1601 2:00:00, FileAttributes: ANCI"
"CloseFile","objects.bin","SUCCESS",""
"CreateFile","objects.bin","SUCCESS","Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
"QueryAttributeTagFile","objects.bin","SUCCESS","Attributes: ANCI, ReparseTag: 0x0"
"SetDispositionInformationFile","objects.bin","SUCCESS","Delete: True"
"CloseFile","objects.bin","SUCCESS",""
"CreateFile","objects.bin","SUCCESS","Desired Access: Generic Read/Write, Disposition: OpenIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0, OpenResult: Created"
"QueryStandardInformationFile","objects.bin","SUCCESS","AllocationSize: 0, EndOfFile: 0, NumberOfLinks: 1, DeletePending: False, Directory: False"
"WriteFile","objects.bin","SUCCESS","Offset: 0, Length: 62, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 62, Length: 70, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 132, Length: 63, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 195, Length: 64, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 259, Length: 63, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 322, Length: 70, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 392, Length: 63, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 455, Length: 64, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 519, Length: 70, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 589, Length: 63, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 652, Length: 64, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 716, Length: 63, Priority: Normal"
"WriteFile","objects.bin","SUCCESS","Offset: 779, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 842, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 905, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 968, Length: 63"
"ReadFile","objects.bin","END OF FILE","Offset: 2 452, Length: 2"
"QueryStandardInformationFile","objects.bin","SUCCESS","AllocationSize: 4 096, EndOfFile: 1 031, NumberOfLinks: 1, DeletePending: False, Directory: False"
"WriteFile","objects.bin","SUCCESS","Offset: 1 031, Length: 64"
"WriteFile","objects.bin","SUCCESS","Offset: 1 095, Length: 70"
"WriteFile","objects.bin","SUCCESS","Offset: 1 165, Length: 70"
"WriteFile","objects.bin","SUCCESS","Offset: 1 235, Length: 64"
"WriteFile","objects.bin","SUCCESS","Offset: 1 299, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 1 362, Length: 64"
"WriteFile","objects.bin","SUCCESS","Offset: 1 426, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 1 489, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 1 552, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 1 615, Length: 64"
"WriteFile","objects.bin","SUCCESS","Offset: 1 679, Length: 70"
"WriteFile","objects.bin","SUCCESS","Offset: 1 749, Length: 64"
"WriteFile","objects.bin","SUCCESS","Offset: 1 813, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 1 876, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 1 939, Length: 63"
"WriteFile","objects.bin","SUCCESS","Offset: 2 002, Length: 63"
"CloseFile","objects.bin","SUCCESS",""

As you see, it writes 16 buffers at offsets 0, 62, 132, ... up to 968 and closes the file! Next it opens it again and proceeds at offsets 1031,1095,1165, which come from another session, where I do the same: open raf and fc, and write another series of buffers, which breaks after 16 buffers written again. The second series starts at 1031 where file was closed and where it was a location for 17th buffer from first write. I see nothing about 16 buffer limitation in the jdocs.

Fortunately, fileChannel.force does not help to cure this situation because I do not need my data physically on disk unless system shuts down. I am happy with data flushed into OS cache at fc.write time or when JVM closes down so that it is available next time that I open the file in the same system.

Here is the program to reproduce the bug

object FileChannelFailureDemo extends App {

  import java.nio._, java.io._, java.nio.channels._

  val raf = new RandomAccessFile("""objects2""", "rw") ; val fc = raf.getChannel
  val range = (1 to 20)
  val ba = range.map (i => ByteBuffer.wrap(Array.ofDim[Byte](i)))
  fc.write(ba.toArray)                      //> res0: Long = 136

  val expectedLength = range.foldLeft(0){case (acc, i) => acc + i}
                                                  //> expectedLength  : Int = 210
  // 1+2+..+20 = 20*21/2 = 210 // epxected len
  // 1+2+..+16 = 16*17/2 = 136 // len of file if only 16 buffers written
  // assertion here, file size 136 != 210                                                 
  assert(raf.length == expectedLength , raf.length + " != " + expectedLength)
                                                  //> java.lang.AssertionError: assertion failed: 136 != 210
                                                  //|   at scala.Predef$.assert(Predef.scala:165)
  println("succeeded, file size is " + raf.length)
  fc.close; raf.close
}

Curiously, but it is not reproduced in that remote machine. I also do not see it if run this short program from console but this code fails assertion executed from Worksheet. It seems to depend on the program state. It can fail or succeed with the same JRE.

The documentation clearly states that not all the requested bytes may be written, and the method returns the number of bytes that were. You have to loop. — user207421, Dec 08 '15 at 13:11
Thanks. But what is the point of this method in this case? It would be a hell to loop using it. I guess that this method could implement the loop calling `write(ByteBuffer[] srcs, int offset, int length)` itself. Otherwise, it is easier to me to implement the loop manually using bufs.foreach.write(_) rather than to to start with write(bufs). What is the point of gather function in this case? — Valentin Tihomirov, Dec 08 '15 at 16:58

Loosing data with (GatheringByteChannel) RandomAccessFile.getChannel.write(buffers)

0 Answers0