-1

I have this:

In [1]:a = sc.parallelize([a,b,c])
In [2]:a.cartesian(a).collect()
Out[3]: [(a, a), (a, b), (a, c), (b, a), (c, a), (b, b), (b, c), (c, b), (c, c)]

I want the following result:

In [1]:a = sc.parallelize([1,2,3])
In [2]:a.cartesianMoreInteligent(a).collect()
Out[3]: [(a, a), (a, b), (a, c), (b, b), (b, c), (c, c)]

Because my calculus return a symetrical matrix (correlation). What is the best way to achieve this ? (No loop) With a, b and c can be anything, even tuple.

GwydionFR
  • 787
  • 1
  • 10
  • 25

1 Answers1

0

Not sure about the python syntax, but in scala you could write:

a.cartesian(a).filter{ case (a,b) => a <= b }.collect()

My guess is in python it would be something like:

a.cartesian(a).filter(lambda a, b: a <= b).collect()

Daniel B.
  • 929
  • 4
  • 8
  • Sorry my example was a bit out of scope, I currently work with string and list it is not possible to order them – GwydionFR May 04 '16 at 12:42