Python List of tuples in Scala

Question

I am using Jython execute the python code part (a python module with utility functions from existing codebase) that returns a list of tuples, but what I get in scala is a simple flattened list. Any suggestions on the cause would help. Since I am a beginner with with Scala and Jython, this probably might not be the best approach to solve the problem. I call the python function as shown below:

val viaJython = true
val interp = new PythonInterpreter()
val pyCode =
  interp.compile(
    """import myModule as df
       | aList = df.find_date(foundVal)"""
  )

interp.set("foundVal", foundVal)
interp.exec(pyCode)
println(interp.get("aList"))

score 1 · Accepted Answer · edited May 23 '17 at 10:27

A simple solution may be: If you get a flattened List and you know the tuple size n then it can be unflattened into a List of Lists with List.grouped(n).toList. Then the sublists can be converted to tuples with map as shown below or using methods given at Convert a Scala list to a tuple? and Is there way to create tuple from list(without codegeneration)?. If you do not know the tuple length then you can find out by examining the bytecode generated by javap -c on the class file.

Another method is to use the implicit conversions from Java collections, iterators, iterables and enumerators which Scala provides. To apply them add "import scala.collection.JavaConversions._" before executing Jython and set resulting Jython List to Scala mutable.Buffer with explicit type declaration. For example, if the Jython List elements are 3-tuples of Int (Tuple3[Int,Int,Int] in Scala) then the converted Scala collection could be defined as:

val pyBuffer: scala.collection.mutable.Buffer[Tuple3[Int,Int,Int]] = ResultingJavaListFromJython

The reason for using scala.collection.mutable.Buffer is that is the collection supported by scala.collection.JavaConversions for Java List conversion. After the conversion to mutable.Buffer is done, it can be converted to any of a number of other types of collections with its "to" functions including toList to convert it to a Scala List, e.g.:

val pyList = pyBuffer.toList

For reference see http://www.scala-lang.org/api/2.11.7/#scala.collection.JavaConversions$ or the equivilant API page for whatever version of Scala you are using.

Another issue is that Java does not have tuples so Jython implements PyTuple with java.util.List and Scala does not provide conversion to Tuple. For that reason another thing to try, assuming that each PyTuple has Int elements, is:

import scala.collection.mutable.Buffer
val pyBuffer2: Buffer[Buffer[Int]] = ResultingJavaListFromJython

Then the elements of pyBuffer2 can be mapped to tuples. For example, assuming each Buffer[Int] element has 3 elements:

import scala.collection.mutable.ArrayBuffer
val pyBuffer3 = pyBuffer2.map{case ArrayBuffer(a,b,c) => (a,b,c)}

Now pyBuffer3 can be converted to Scala List with toList as shown above.

The reason for importing scala.collection.mutable.ArrayBuffer and matching on it is that it is Scala's default implementation of mutable.ArrayBuffer which is a trait.

Thanks for the excellent information on the issue. The approach makes sense. A minor issue though is that it seems that Scala does not implicitly coerce the types from PyObject returned from "ResultingJavaListFromJython", unless I missed something obvious. — 4rk, Jul 01 '15 at 06:17
What exactly do you mean? Exactly what is the structure of the PyObject including types? The purposed of JavaConversions is to converts Java collections, iterators, iterables and enumerables to corresponding Scala types. It is possible to implement custom explicit or implicit conversions when needed and if sensible. A hitch may be that you need to know exactly what you are converting from to convert it. — , Jul 01 '15 at 20:09
The structure of the object returned from python code is List of (String, String). If I explicitly typecast the object as `interp.get("aList").asInstanceOf[PyList]`, it works fine, and I get the flattened list (as does Java.util.List), but mapping `alist` to `pybuffer2` as shown above ends into type mismatch (Of course, after changing `Int` to `String`) — 4rk, Jul 01 '15 at 20:35
See the 1st paragraph of my answer where I added a possible simple solution of unflattenning the List you get and converting its sublists to tuples. — , Jul 01 '15 at 20:35
Glad to hear it. Mapping alist to pybuffer2 does not work as shown since alist is not a list of lists. Maybe it would work if instead of using interp.get("aList").asInstanceOf[PyList] it was typecast as interp.get("aList").asInstanceOf[PyList[PyList]] if that's possible. The latter is preferable since its preserves the nested structure of a list of tuples while recognizing that Jython tuples are implemented as lists. — , Jul 01 '15 at 20:55
Well, I did try that (interp.get("aList").asInstanceOf[PyList[PyList]] ), but since Pylist does not take type parameters we get an error on nesting types. — 4rk, Jul 01 '15 at 21:20

Python List of tuples in Scala

1 Answers1