0

I'm building a feed where a user can follow projects, and those projects will show up in the user's feed, sorted by a lastUpdated field, and allow for pagination, but I can't figure out what the best approach is.

The basic Firestore structure looks like:

users (collection)
  projects (subcollection)
    lastUpdated (timestamp)
  followingProjects (array of project Document IDs)

I could use an in operator like:

let followingProjects = user.followingProjects
firestore().collectionGroup("projects")
           .whereField(FieldPath.documentID(), in: followingProjects)
           .order(by: "lastUpdated", descending: true)
           .limit(to: 20)
           // And later for pagination :
           .start(afterDocument: startAfter)

But I believe the in operator is limited to 30 values in followingProjects, and I would like to allow following more than 30 projects.

A solution I can think of is to create a separate subcollection for followingProjects, so the new structure would be:

users (collection)
  projects (subcollection)
    lastUpdated (timestamp)
  followingProjects (subcollection)
    id (string)
    lastUpdated (timestamp)

In this case, to get the feed, I'd run a collectionGroup query to get the latest followingProjects so I can sort and paginate, followed by individual getDocument() queries for each project. Every time a project is updated, it would have to update the lastUpdated for each user that's following the project though, and this seems inefficient.

Is there a better approach I'm missing?

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
Cody
  • 650
  • 9
  • 16
  • 1
    You also can't use `FieldPath.documentID()` as a filter in a collection group query. You can only filter on document fields, and the document ID isn't considered one. https://stackoverflow.com/questions/56188250/how-to-perform-collection-group-query-using-document-id-in-cloud-firestore – Doug Stevenson Jul 16 '23 at 17:18
  • @DougStevenson Oh thanks for the clarification, I was wondering about that! – Cody Jul 16 '23 at 17:34
  • "this seems inefficient" What is inefficient about it? What would you consider efficient for that? – Frank van Puffelen Jul 16 '23 at 21:09
  • It seems inefficient in terms of the number of write operations to support paginating the most recently updated Projects. If a project has 1000 followers, any update to the project would require 1000 writes, one for each follower, to update its lastUpdated field (the lastUpdated field need to be updated so the followingProjects collectionGroup query can be ordered and paginated). The end goal is for the user to have a pagination-enabled feed that shows the most recently updated Projects that they're following, with no limit on how many Projects they can follow. – Cody Jul 16 '23 at 22:04

2 Answers2

0

In a collection group index, the __name__ field (which is what FieldPath.documentID() maps to) is populated with the path of the document, rather than just its document ID. So if you want to filter on FieldPath.documentID(), you will need to know the entire path of the document(s).

Alternatively, you can add a regular field with the document ID to each document, and filter on that in your collection group query.

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
  • That would work, then I can do whereField("documentID", in: followingProjects)! But my main issue is that [according to this limit on disjunctions](https://firebase.google.com/docs/firestore/query-data/queries?hl=en&authuser=0#limits_on_or_queries), it would be limited to 30 entries in followingProjects. I could feed in 30 projects at a time, but then I can't use the sort order or pagination. I'm hoping to support possibly hundreds of projects to follow. – Cody Jul 16 '23 at 17:55
  • If you want to get more than 30 values, you'll have to use multiple queries. But that is unrelated to using a single top-level collection or multiple subcollections. – Frank van Puffelen Jul 16 '23 at 18:17
  • I think the OP was asking specifically how to avoid having to get the individual documents *after* the collection group query. The fact that they were using an invalid filter on the query was incidental (hence my comment). The primary solution here is (of course and unfortunately) to copy all the other document data into the documents that participated in the collection group query. – Doug Stevenson Jul 16 '23 at 18:44
  • Right! The added reads to the individual documents and syncing all of the user's followingProjects lastUpdated timestamp when a Project is updated. So if a Project has 10 followers, then writing to a Project now has to write to 10 other documents in every user's followingProjects subcollection. I'm open to reorganizing the collection hierarchy, if it helps, but I just can't think of how it would help. Thank you! – Cody Jul 16 '23 at 19:54
0

I found the answer I needed in another answer!

https://stackoverflow.com/a/58100387/482536

Instead of storing Project IDs in a followingProjects array as part of the User, the Project should store User IDs in a followers array. Then I can run the following query without worrying about the disjunctions limit, and there are no separate documents to keep updated whenever a Project is updated.

let userID = user.id
firestore().collectionGroup("projects")
       .whereField("followers", arrayContains: userID)
       .order(by: "lastUpdated", descending: true)
       .limit(to: 20)
       // And later for pagination :
       .start(afterDocument: startAfter)
Cody
  • 650
  • 9
  • 16