Extract range of bytes from file in Scala

Question

I have a binary file that I need to extract some range of bytes from: start: Long - end: Long. I need Long because there are several gigagbytes. My app needs to give the result back as a ByteString. I tried

val content: Array[Byte] = Array()
val stream: FileInputStream = new FileInputStream(file: File)
stream.skip(start)
stream.read(content, 0, end-start)

but already I cannot use Long in read, only Int (is this a bug? skip is ok with Long...). Also I would need to convert the result to ByteString. I would love to do this, too:

val stream: FileInputStream = new FileInputStream(file: File)
stream.skip(start)
org.apache.commons.io.IOUtils.toByteArray(stream)

but how do I tell it where to end? stream has no method takeWhile or take. Then I tried

val source = scala.io.Source.fromFile(file: File)
source.drop(start).take(end-start)

Again, only Int in drop...

How can I do that ?

http://stackoverflow.com/questions/28883876/can-i-do-a-lazy-take-with-a-long-parameter — Yuval Itzchakov, Jul 28 '16 at 15:24
With that I was trying to implement a correct response to a byte-range request in Play, but since recently Play seems to have implemented that, and documents it in "Working with public assets"... — JulienD, Jul 28 '16 at 17:31

Dylan · Accepted Answer · 2016-07-28T17:24:43.093

Use IOUtils.toByteArray(InputStream input, long size)

val stream = new FileInputStream(file)
stream.skip(start)
val bytesICareAbout = IOUtils.toByteArray(stream, end-start)
// form the ByteString from bytesICareAbout

Note this will throw if end - start is greater than Integer.MAX_VALUE, for a good reason! You wouldn't want a 2GB array to be allocated in-memory.

If for some reason your end - start > Integer.MAX_VALUE, you should definitely avoid allocating a single ByteString to represent the data. Instead, you should do something like:

import org.apache.commons.io.input.BoundedInputStream

val stream = new FileInputStream(file)
stream.skip(start)
val boundedStream = new BoundedInputStream(stream, start - end)

Oh, I did not see that there was a second constructor, thanks, it works ! — JulienD, Jul 28 '16 at 17:29

Extract range of bytes from file in Scala

1 Answers1