15

I would like to create a Row with multiple arguments without knowing their number. I wrote something like this in Scala:

def customRow(balance: Int,
              globalGrade: Int,
              indicators: Double*): Row = {
    Row(
      balance,
      globalGrade,
      indicators:_*
    )
}

On Spark GitHub, the Row object seems to accept the :_* notation considering its apply method:

def apply(values: Any*): Row = new GenericRow(values.toArray)

But at compilation time, this doesn't seem to be allowed:

Error:(212, 19) no ': _*' annotation allowed here
(such annotations are only allowed in arguments to *-parameters)
        indicators:_*

What did I miss?

Jacek Laskowski
  • 72,696
  • 27
  • 242
  • 420
Baptiste Merliot
  • 841
  • 11
  • 24

2 Answers2

22

This minimal example may explain better why what you want to do is not allowed:

def f(a: Int, b: Int, c: Int, rest: Int*) = a + b + c + rest.sum

val ns = List(99, 88, 77)

f(11, 22, 33, 44, ns:_*) // Illegal
f(11, 22, 33,     ns:_*) // Legal
f(11, 22,         ns:_*) // Illegal

Basically, you can use the :_* syntax only to pass a sequence directly as the vararg parameter rest, but it's all-or-nothing. The sequence's items are not shared out between simple and vararg parameters, and the vararg parameter cannot gather values from both the simple arguments and the provided sequence.

In your case, you are trying to call Row as if it had two simple parameters and then a vararg one, but that's not the case. When you create the sequence yourself, you are making it fit correctly into the signature.

Note that in dynamically-typed programming languages, this is typically not an issue. For example, in Python:

>>> def f(a, b, c, *rest):
    return a + b + c + sum(rest)

>>> ns = [99, 88, 77]
>>> f(11, 22, 33, 44, *ns)
374
>>> f(11, 22, 33, *ns)
330
>>> f(11, 22, *ns)
297

Update (2023-07-20)

It seems that mixed variable arguments will be supported soon.

Roberto Bonvallet
  • 31,943
  • 5
  • 40
  • 57
7

Resolved it by adding an intermediary Seq :

def customRow(balance: Int,
              globalGrade: Int,
              indicators: Double*): Row = {

  val args = Seq(
    balance,
    globalGrade
  ) ++ indicators

    Row(
      args:_*
    )
}

But still, I do not know why it works.

Baptiste Merliot
  • 841
  • 11
  • 24