Efficiently compare all combinations of two in a Scala collection

Question

Working with OpenFlow I want to compute how many pairs of flows satisfy the following condition (Given any two flows, for example f1 and f2):

f1's Src ip must be equal to f2's Dst ip.
f1's Dst ip must be equal to f2's Src ip.
f1 and f2 must have the same protocol.

This I think would be a combination of two elements (NFlows Choose 2). Thanks to this question I am using the method combinations like below:

val pairFlows = matchs.combinations(2).filter{ list =>
  val f1 = list.head
  val f2 = list.tail.head

  f1.dl_type == f2.dl_type &&
  f1.nw_dst == f2.nw_src &&
  f1.nw_src == f2.nw_dst
}

My question is, Is this the most efficient way of performing this computation? Or would it be better to create a HashMap in order to keep track of each src, dst and protocol type?

Here is an example of the Data:

{
    1: [
        {
            "actions": [
                "OUTPUT:2"
            ],
            "idle_timeout": 0,
            "cookie": 0,
            "packet_count": 18,
            "hard_timeout": 0,
            "byte_count": 1764,
            "duration_sec": 6884,
            "duration_nsec": 5000000,
            "priority": 1,
            "length": 120,
            "flags": 0,
            "table_id": 0,
            "match": {
                "dl_type": 2048,
                "dl_dst": "00:00:00:00:00:02",
                "nw_src": "10.0.0.1",
                "in_port": 1,
                "nw_dst": "10.0.0.2"
            }
        },
        {
            "actions": [
                "OUTPUT:2"
            ],
            "idle_timeout": 0,
            "cookie": 0,
            "packet_count": 18,
            "hard_timeout": 0,
            "byte_count": 1764,
            "duration_sec": 1123,
            "duration_nsec": 181000000,
            "priority": 1,
            "length": 120,
            "flags": 0,
            "table_id": 0,
            "match": {
                "dl_type": 2048,
                "dl_dst": "00:00:00:00:00:02",
                "nw_src": "10.0.0.3",
                "in_port": 3,
                "nw_dst": "10.0.0.2"
            }
        },
        {
            "actions": [
                "OUTPUT:1"
            ],
            "idle_timeout": 0,
            "cookie": 0,
            "packet_count": 10,
            "hard_timeout": 0,
            "byte_count": 980,
            "duration_sec": 1132,
            "duration_nsec": 253000000,
            "priority": 1,
            "length": 120,
            "flags": 0,
            "table_id": 0,
            "match": {
                "dl_type": 2048,
                "dl_dst": "00:00:00:00:00:01",
                "nw_src": "10.0.0.3",
                "in_port": 3,
                "nw_dst": "10.0.0.1"
            }
        },
        {
            "actions": [
                "OUTPUT:1"
            ],
            "idle_timeout": 0,
            "cookie": 0,
            "packet_count": 16,
            "hard_timeout": 0,
            "byte_count": 1568,
            "duration_sec": 1141,
            "duration_nsec": 652000000,
            "priority": 1,
            "length": 120,
            "flags": 0,
            "table_id": 0,
            "match": {
                "dl_type": 2048,
                "dl_dst": "00:00:00:00:00:01",
                "nw_src": "10.0.0.2",
                "in_port": 2,
                "nw_dst": "10.0.0.1"
            }
        },
        {
            "actions": [
                "OUTPUT:3"
            ],
            "idle_timeout": 0,
            "cookie": 0,
            "packet_count": 20,
            "hard_timeout": 0,
            "byte_count": 1960,
            "duration_sec": 6883,
            "duration_nsec": 961000000,
            "priority": 1,
            "length": 120,
            "flags": 0,
            "table_id": 0,
            "match": {
                "dl_type": 2048,
                "dl_dst": "00:00:00:00:00:03",
                "nw_src": "10.0.0.2",
                "in_port": 2,
                "nw_dst": "10.0.0.3"
            }
        },
        {
            "actions": [
                "OUTPUT:3"
            ],
            "idle_timeout": 0,
            "cookie": 0,
            "packet_count": 12,
            "hard_timeout": 0,
            "byte_count": 1176,
            "duration_sec": 6883,
            "duration_nsec": 984000000,
            "priority": 1,
            "length": 120,
            "flags": 0,
            "table_id": 0,
            "match": {
                "dl_type": 2048,
                "dl_dst": "00:00:00:00:00:03",
                "nw_src": "10.0.0.1",
                "in_port": 1,
                "nw_dst": "10.0.0.3"
            }
        },
        {
            "actions": [
                "OUTPUT:CONTROLLER"
            ],
            "idle_timeout": 0,
            "cookie": 0,
            "packet_count": 0,
            "hard_timeout": 0,
            "byte_count": 0,
            "duration_sec": 9156,
            "duration_nsec": 252000000,
            "priority": 65535,
            "length": 96,
            "flags": 0,
            "table_id": 0,
            "match": {
                "dl_type": 35020,
                "dl_dst": "01:80:c2:00:00:0e"
            }
        },
        {
            "actions": [
                "OUTPUT:CONTROLLER"
            ],
            "idle_timeout": 0,
            "cookie": 0,
            "packet_count": 22,
            "hard_timeout": 0,
            "byte_count": 1260,
            "duration_sec": 9156,
            "duration_nsec": 268000000,
            "priority": 0,
            "length": 80,
            "flags": 0,
            "table_id": 0,
            "match": {}
        }
    ],
}

Here there are 8 flows, 3 of which are pairs.

Starting with a groupBy(_.dl_type) would give you several smaller lists instead of one huge, to check. Depending on how your data looks, this could help alot, a little or not at all. Also, looking through all combinations of 2 elements from a list of n elements is O(n^2). If you instead keep two lists, one sorted by dst and one by src, (this is done in O(nlogn)), you can then traverse them once and collect all matches. This will give you better time complexity than the example above. — Buhb, Jan 16 '17 at 16:16

Alejandro Alcalde · Accepted Answer · 2017-01-17T12:07:27.877

I finally wrote the code to be O(n), I keep a Map with key f.nw_src + f.nw_dst + f.dl_type and value an Int representing if that key has the corresponding pair flow (Which would be the one with key f.nw_dst + f.nw_src + f.dl_type. Here is the final code:

val table =  flows./:(Map.empty[String,Int]){ case (m,f) =>
  val key = f.nw_src + f.nw_dst + f.dl_type
  val inverseKey = f.nw_dst + f.nw_src + f.dl_type
  val haspair = m get inverseKey match {
    case Some(v) => v + 1
    case None => 0
  }
  m + (key -> haspair)
}

val pairs = table.filter(_._2 > 0)

Hope this help someone looking for a similar question.

Efficiently compare all combinations of two in a Scala collection

1 Answers1