Optimize this query to get lineage diagram in graph using gremlin

Question

I have my data set up in cosmos DB gremlin API. I am trying to write a query that can starting at a given vertex, gives all the edges with labels and properties in both directions, and the vertices with their properties, with a loop count of lets say 2. I am running the following queries.

Data to be setup

g.addV('item').property('id','1').property('name', 'item1')
g.addV('item').property('id','2').property('name', 'item2')
g.addV('item').property('id','3').property('name', 'item3')
g.addV('item').property('id','4').property('name', 'item4')
g.addV('item').property('id','5').property('name', 'item5')
g.addV('item').property('id','6').property('name', 'item6')
g.addV('item').property('id','7').property('name', 'item7')

g.addV('report').property('id','8').property('name', 'report1')
g.addV('report').property('id','9').property('name', 'report2')
g.addV('folder').property('id','10').property('name', 'folder1')
g.addV('server').property('id','11').property('name', 'server1')

g.V('1').addE('hasParent').to(g.V('2'))
g.V('1').addE('hasParent').to(g.V('3'))
g.V('2').addE('hasParent').to(g.V('4'))
g.V('5').addE('hasParent').to(g.V('6'))
g.V('4').addE('hasParent').to(g.V('5'))
g.V('7').addE('hasParent').to(g.V('5'))

g.V('8').addE('hasParent').to(g.V('9'))
g.V('8').addE('hasParent').to(g.V('9'))
g.V('8').addE('belongsTo').to(g.V('10'))
g.V('10').addE('belongsTo').to(g.V('11'))

query: g.V('id','1').repeat(bothE().dedup().otherV()).times(2).path()

response:

[
  {
    "labels": [
      [],
      [],
      [],
      [],
      []
    ],
    "objects": [
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      },
      {
        "id": "a6b02073-68f8-4d53-8af0-54739eaaeaa0",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "2",
        "outV": "1"
      },
      {
        "id": "2",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "2|name",
              "value": "item2"
            }
          ]
        }
      },
      {
        "id": "c8722748-3899-4a2c-afe4-6f13b6c1f8d5",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "4",
        "outV": "2"
      },
      {
        "id": "4",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "4|name",
              "value": "item4"
            }
          ]
        }
      }
    ]
  },
  {
    "labels": [
      [],
      [],
      [],
      [],
      []
    ],
    "objects": [
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      },
      {
        "id": "a6b02073-68f8-4d53-8af0-54739eaaeaa0",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "2",
        "outV": "1"
      },
      {
        "id": "2",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "2|name",
              "value": "item2"
            }
          ]
        }
      },
      {
        "id": "a6b02073-68f8-4d53-8af0-54739eaaeaa0",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "2",
        "outV": "1"
      },
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      }
    ]
  },
  {
    "labels": [
      [],
      [],
      [],
      [],
      []
    ],
    "objects": [
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      },
      {
        "id": "2f0c5890-3fa5-4d7d-8ada-910e5419750f",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "3",
        "outV": "1"
      },
      {
        "id": "3",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "3|name",
              "value": "item3"
            }
          ]
        }
      },
      {
        "id": "2f0c5890-3fa5-4d7d-8ada-910e5419750f",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "3",
        "outV": "1"
      },
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      }
    ]
  }
]

This is giving me the desired output, but has a lot of repetitions, i am guessing because it prints the path as it traverses. How can I modify this query to return the smallest possible json output? Just return the vertices and edges once. I am using cosmosDB gremlin APIs if that makes any difference.

Response needed: just a list of vertices and edges needed for painting this graph.

stephen mallette · Accepted Answer · 2020-07-26T13:02:40.957

3

Your example as-is doesn't produce the same output that I see in your question - I get just one path:

gremlin> g.V('id','1').repeat(bothE().dedup().otherV()).times(2).path()
==>[v[1],e[11][1-hasParent->2],v[2],e[13][2-hasParent->4],v[4]]

I'm not sure if that's some mistake in your data script but I think I get the gist of what you are asking from your output. I can recreate a situation of duplication by doing an emit() on the repeat() to include all paths in the output:

gremlin> g.V('id','1').repeat(bothE().dedup().otherV()).emit().times(2).path()
==>[v[1],e[11][1-hasParent->2],v[2]]
==>[v[1],e[12][1-hasParent->3],v[3]]
==>[v[1],e[11][1-hasParent->2],v[2],e[13][2-hasParent->4],v[4]]

Now I have some duplication of vertices/edges in the resulting output. If you just want a unique list of all those objects, just unfold() each Path and dedup():

gremlin> g.V('id','1').repeat(bothE().dedup().otherV()).emit().times(2).path().unfold().dedup()
==>v[1]
==>e[11][1-hasParent->2]
==>v[2]
==>e[12][1-hasParent->3]
==>v[3]
==>e[13][2-hasParent->4]
==>v[4]

This is a bit of an interesting pattern in a sense because it provides an easy way to produce a subgraph for Gremlin Language Variants and graph systems that don't support subgraph() step.

edited Jul 26 '20 at 13:02

answered Jul 24 '20 at 10:47

stephen mallette

45,298
5
67
135

This doesn't give the desired results, atleast not in cosmosDB. I have reduced the repeat step to 1 to show what i mean. `g.V('id','POLICE OFFICER').repeat(bothE().dedup().otherV()).times(1).path().limit(10)`. I am adding the response to the question above to provide clarity on what i mean when i say duplicate – Michael Scott Jul 24 '20 at 20:19
Could you please provide a Gremlin script that creates some sample data - here is an example https://stackoverflow.com/questions/51388315/gremlin-choose-one-item-at-random – stephen mallette Jul 25 '20 at 01:01
I edited my question to create some sample data in cosmos, the query i am making and the response i am getting. I hope this is clearer than earlier – Michael Scott Jul 26 '20 at 03:39
updated my answer - i hope i understood your issue this time. – stephen mallette Jul 26 '20 at 13:02

Optimize this query to get lineage diagram in graph using gremlin

1 Answers1