0

I am inserting documents into a collection with Firebase using monotonically increasing alphabetically sortable IDs (that are RFC3339-formatted dates). This helps me query the oldest documents and remove them when I reach a certain number.

So my doc IDs look like these strings:

  • "2021-05-21T18:00:00Z"
  • "2021-05-21T21:00:00Z"
  • "2021-05-21T22:00:00Z"

I am using cloud.google.com/go/firestore Go library to retrieve docs with IDs less than (<) a particular value (then I plan on deleting these docs in a batch operation, e.g. delete docs older than 30 days).

However, the Go package for this returns no results for my query (at least on the Firebase Emulator):

    iter := u.DB.Collection("users").Doc(uid).Collection("orders").
        Where(firestore.DocumentID, "<", "2020-04-23T02:00:00Z").Documents(ctx)

    // iter comes back empty.

I cannot tell if it is a limitation of the product (e.g. __name__ does not support querying with < or >) or if it is a shortcoming/unimplemented behavior of the Firebase Emulator.

This question probably should be about why this doesn't work ––but if anyone has suggestions around keeping max N records in a collection in a rotating fashion (like a FIFO queue with a cap) I am interested in hearing as this is why I am trying to use this query.

ahmet alp balkan
  • 42,679
  • 38
  • 138
  • 214
  • Have you considered using an `orderBy` and `StartAt`: ```query := u.DB..Collection("users").Doc(uid).Collection("orders").OrderBy(firestore.DocumentID, firestore.Asc).StartAt("2020-04-23T02:00:00Z")``` Not sure if the code is OK. Never used Go -.- You can find more about it here: https://firebase.google.com/docs/firestore/query-data/query-cursors – Tarik Huber May 22 '21 at 21:43
  • Other factor: \_\_name\_\_ does not do what you THINK it does. see https://stackoverflow.com/questions/56149601/firestore-collection-group-query-on-documentid/58104104#58104104 . It actually is the FULL PATH INCLUDING THE documentId, and has to be matched as such. – LeadDreamer May 22 '21 at 23:11
  • BTW, using such ordered documentID's is **HIGHLY HIGHLY HIGHLY** discouraged - it WILL impact database performance. – LeadDreamer May 22 '21 at 23:13

2 Answers2

0

Other factor: __name__ does not do what you THINK it does. see Firestore collection group query on documentId . It actually is the FULL PATH INCLUDING THE documentId, and has to be matched as such.

BTW, using such ordered documentID's is HIGHLY HIGHLY HIGHLY discouraged - it WILL impact database performance. You will run into contention and collision on both reads and writes. Put these values into a field in the documents, and just use random Id's (such as created by .doc()) as the document Id's. That way you can search, sort, limit and anything else you want on the data. There is virtually NEVER a good reason to use ordered, regular Id's for documents.

LeadDreamer
  • 3,303
  • 2
  • 15
  • 18
0

To achieve this in a scalable format, you should maintain an indexable field inside each document that is its document ID, which appears to be a timestamp. from there, you can easily query and sort based on your needs.

This is because scraping the firestore collection to find by ID impacts read performance on Firestore, leading to long waiting times that enhance with each document added, it's much more ideal and scalable to use the index.

As for the comments about Collection Group Queries: If you are limiting your query to only 1 specific user's "orders" collection, you do not need a collection group query as this searches amongst ALL collections called "orders".

DIGI Byte
  • 4,225
  • 1
  • 12
  • 20
  • My question was more like: Why doesn’t this query operator work (returns no results). And I assume __name__ is already sorted with an ascending index at Firestore (because queries return docs sorted by ID by default). – ahmet alp balkan May 23 '21 at 02:03
  • because the names of the document are not queryable, they are returned when listing and paginating. but for all intents - are not monitored by the search index. – DIGI Byte May 23 '21 at 03:29
  • doc IDs are I think queryable with operators like =, In, NotIn. – ahmet alp balkan May 23 '21 at 07:53
  • you might want to instead paginate your results specified by a document snapshot and setting a limit https://cloud.google.com/firestore/docs/query-data/query-cursors#use_a_document_snapshot_to_define_the_query_cursor – DIGI Byte May 23 '21 at 08:43
  • It may be possible to query by document metadata, but to date, this is not documented or practiced within the community. the common assumption is that metadata is only available on the snapshot after a request has been resolved. – DIGI Byte May 24 '21 at 02:09