0

Is it possible to query Parent + Child entities in a single query pass?

This would require a JOIN in SQL, but in the underlying BigTable the keys are sequential, so it should theoretically by possible to scan Parent and Child entities in a single pass. For example:

  • Parent-1
  • Parent-1-Child-A
  • Parent-1-Child-B
  • Parent-1-Child-C
  • Parent-2
  • Parent-2-Child-D
  • Parent-3
  • Parent-3-Child-E

Ancestor queries limit the scope to a single Parent. What I'm looking for is Query by key range that isn't limited to a single Kind.

Community
  • 1
  • 1
rsb
  • 1,020
  • 1
  • 10
  • 25

1 Answers1

0

Something like that doesn't exist by default.

One way I have addressed this is by having each entity store all of it's possible paths as a repeated property, including itself if you want it to be returned in the query the way ancestor queries do. In addition store the path of the immediate parent.

ie (I will include an example with more than 2 levels)

Parent-1 
paths = ['Parent-1']
parent = []

Parent-1-Child-A
paths = ['Parent-1','Parent-1/Child-A']
parent = ['Parent-1']

Parent-1-Child-C
paths = ['Parent-1','Parent-1/Child-C']
parent = ['Parent-1']

Parent-1-Child-C-Child-F
paths = ['Parent-1','Parent-1/Child-C','Parent-1/Child-C/Child-F' ]
parent = ['Parent-1/Child-C']

Parent-1-Child-C-Child-E
paths = ['Parent-1','Parent-1/Child-C','Parent-1/Child-C/Child-E' ]
parent = ['Parent-1/Child-C']

This way you can query for any key range and limit the depth to immediate children) Ancestor queries have no means of limiting depth.

This would require you to use PolyModel (you haven't said if your using python or java - I don't know if java has a PolyModel analog). So Parent and Child would inherit from Node which would be based on PolyModel

class Node(ndb.PolyModel):
  pass

class Parent(Node):
  pass

class Child(Node):
  pass

Though you may not need different classes for parent and child.

One thing to note, if you use ancestors/parents in the key you can not re-arrange your heirarchies without completely copying/re-writing all of the children

Tim Hoffman
  • 12,976
  • 1
  • 17
  • 29
  • Thanks Tim, that makes sense, although it does force a single Kind (or at least inherited Kind) hierarchy, which I was trying to avoid. Seems like a restriction of NDB rather than the underlying data store. – rsb Jun 02 '14 at 19:52
  • Actually its a limitation of the datastore rather than ndb. All indexes are kind specific. The only kind less query the datstore supports is an ancestor query. – Tim Hoffman Jun 02 '14 at 23:46
  • Tim, I understand indexes are kind specific, but it's my understanding that the underlying BigTable keyspace isn't -- it's composite (/a, /a/b, /a/b/c, /b, etc.) -- so it should be possible to do an efficient key range scan (without using any secondary index). https://developers.google.com/appengine/docs/python/ndb/entities The sequence of entities beginning with a root entity and proceeding from parent to child, leading to a given entity, constitute that entity's ancestor path. – rsb Jun 03 '14 at 20:04
  • Try it. There is no hidden syntax. Notice the limitations of GQL and the datastore api's . There is no mechanism to query across kinds, except with a kindless ancestor query. Just because BigTable might be built a certain way, the datastore is not BigTable but a layer above. I would like to be proven wrong. Try going one level down and work with the raw datastore api. I have used it a bit, and haven't seen any mechanism to do it. – Tim Hoffman Jun 03 '14 at 23:26