Serializing to disk and deserializing Scala objects using Pickling

Question

Given a stream of homogeneous typed object, how would I go about serializing them to binary, writing them to disk, reading them from disk and then deserializing them using Scala Pickling?

For example:

object PicklingIteratorExample extends App {

    import scala.pickling.Defaults._
    import scala.pickling.binary._
    import scala.pickling.static._

    case class Person(name: String, age: Int)

    val personsIt = Iterator.from(0).take(10).map(i => Person(i.toString, i))
    val pklsIt = personsIt.map(_.pickle)

    ??? // Write to disk
    val readIt: Iterator[Person] = ??? // Read from disk and unpickle
}

score 2 · Answer 1 · answered Jun 09 '15 at 13:39

I find a way to so for standard files:

object PickleIOExample extends App {
    import scala.pickling.Defaults._
    import scala.pickling.binary._
    import scala.pickling.static._

    val tempPath = File.createTempFile("pickling", ".gz").getAbsolutePath
    val outputStream = new FileOutputStream(tempPath) 
    val inputStream = new FileInputStream(tempPath) 

    val persons = for{
        i <- 1 to 100
    } yield Person(i.toString, i)

    val output = new StreamOutput(outputStream)
    persons.foreach(_.pickleTo(output))
    outputStream.close()

    val personsIt = new Iterator[Person]{
        val streamPickle = BinaryPickle(inputStream)

        override def hasNext: Boolean = inputStream.available > 0

        override def next(): Person = streamPickle.unpickle[Person]
    }

    println(personsIt.mkString(", "))
    inputStream.close()
}

But I am still unable to find a solution that will work with gzipped files. Since I do not know how to detect the EOF? The following throws an EOFexception since GZIPInputStream available method does not indicate the EOF:

object PickleIOExample extends App {
    import scala.pickling.Defaults._
    import scala.pickling.binary._
    import scala.pickling.static._

    val tempPath = File.createTempFile("pickling", ".gz").getAbsolutePath
    val outputStream = new GZIPOutputStream(new FileOutputStream(tempPath))
    val inputStream = new GZIPInputStream(new FileInputStream(tempPath))

    val persons = for{
        i <- 1 to 100
    } yield Person(i.toString, i)

    val output = new StreamOutput(outputStream)
    persons.foreach(_.pickleTo(output))
    outputStream.close()

    val personsIt = new Iterator[Person]{
        val streamPickle = BinaryPickle(inputStream)

        override def hasNext: Boolean = inputStream.available > 0

        override def next(): Person = streamPickle.unpickle[Person]
    }

    println(personsIt.mkString(", "))
    inputStream.close()
}

Serializing to disk and deserializing Scala objects using Pickling

1 Answers1