4

I have a posts collection in which my application stores all the posts of a user, with the following data structure:

+ uid (the user id of the creator)
+ text-content : String
+ tagged-users: [String]
+ assigned-media: [{"downloadUrl": String, "storageLocation": String}]
+ type: String
+ typestamp: Timestamp
+ Likes
    + uid (the user id of the liking person)
    + timestamp: Timestamp
    + postId (optional, depending on the results of the collection group queries)

Now, I would like to display all the posts of this collection to my user; but only, if the user follows the person who created the post, i.e the uid field of the post document fits.

I know that it is simple to query the collection if the user only follows one person, i.e. I have one simple where clause. But: What if my user follows 10.000 people or more? How can I query that?

Also, how should I store the followings, i.e. the persons a user follows? Using a simple array doesn't seem suitable for me, but with all the other options I can't query the posts collection for posts that come from a person I follow.

I would be grateful for every help I could get!

linus_hologram
  • 1,595
  • 13
  • 38

1 Answers1

3

This is a kind of complex question (and I'm not 100% sure that this is the right place to ask for it), I will do my best to come with something to get you started,

Disclaimer, this might or not for your use case

Divide your service into 3 collections: - users - followers - posts

users Collection is very straight forward, you just need the document id (you already have it).

{
  "andyIsTheBest": {
    uid: `andyIsTheBest`
    // ...
  },
  "sophieTheBest": {
     uid: `sophieTheBest`,
     // ...
   },
   "andyTheWorst": {
     uid: `andyTheWorst`,
     // ...
   },
}

followers mirror the user's docs id, it should look like this:

{
  "andyIsTheBest": {
     last_post: 2019,
     followers: [
       'sophieTheBest', 'andyTheWorst'
     ],
     recent: [
       {
         post_id: 112233, // from the post collection
         seq: 2019
       }
     ]
  }
}

Finally, the posts collection looks something like this:

{
   "112233": {
     seq: 2019,
     body: `blabla the cool post`
     author: `andyIsTheBest`
   }
}

Important, notice the sequence seq number also the last_post, you can update this value via cloud functions so you keep track of it, then the rest are just implementations details:

const remove = firebase.firestore.FieldValue.arrayRemove
const union = firebase.firestore.FieldValue.arrayUnion

const follow = async (toFollow, user) => {
    await db
    .collection('followers')
    .doc(toFollow)
    .update({ users: union(user) });
}

const unfollow  = async (toUnFollow, user) => {
    await db
    .collection('followers')
    .doc(toUnFollow);
    .update({ users: remove(user) });
}

const feed = async (user) => {
   await db.collection(`followers`)
      .where(`followers`, `array-contains`, user)
      .orderBy(`last_post`, `desc`)
      .limit(10)
      .get()

   // do whatever business logic you need down here
}

Basically this way, it doesn't matter if you have 1 or 1 Trillion, the queries should be ok.

There are a lot of details I didn't cover but this should be ok to get you started.

Hope this help.

Credits to my coworkers who implemented this at my company ^^

andresmijares
  • 3,658
  • 4
  • 34
  • 40
  • 1
    Thank you very much for your answer! I really like your approach and it also seems logic to me - but isn' there a 1MB max. document limit in Cloud Firestore? This would become a problem if a user has 1 Mio. followers? – linus_hologram Sep 26 '19 at 19:06
  • you Have 1Mb max per doc... but you don’t to load the whole doc... for example, you can have only an thumbnail and small preview of the text and keep this in sync with the collection where you are storaging the whole posts via cloud functions... playing this with this with pagination will do the trick... it’s not about the query, it mostly about how you orchestrate your app – andresmijares Sep 26 '19 at 19:59
  • this won't work if a user follows 2K people as all following-kid's are stored in one array. Firebase Firestore docs have a limit of 1MB in size and that will be exceeded. – linus_hologram Sep 27 '19 at 14:28
  • do you have any other idea how I could model my data to avoid the 1MB limit? – linus_hologram Oct 03 '19 at 14:22
  • 1
    why won't it work? you don't have to sync all the information, you only need to add on this collection the data needed for your view... also if that limit per doc is too much for you probably firestore is not the right solution for your problem, it's hard for me to imagine which info you need to store for a social media bigger than 1Mb – andresmijares Oct 03 '19 at 15:17
  • 1
    Thanks for your reply. Take a look at the followers collection: in there, you have a field „follower“ of type Array. If a user has 1 million followers, the array would store 1 million user id‘s, correct? However, the 1MB document size limit would not allow the array to get that big. The rest would work. – linus_hologram Oct 03 '19 at 15:20
  • you are mixing up a few stuff, the follower collection only storage data needed for your view, ex: a summary of the last 2 posts... the posts collection holds the whole post information... the limit of 1Mb is for docs, each unless you query all the posts for your user which you should never cause, you should be ok – andresmijares Oct 03 '19 at 15:54
  • 1
    Yes, but what about this part: `followers: [uid1, uid2]` ? In there I would store all user ids of people following me, right? What if 100.000 people follow me? I don’t quite get it – linus_hologram Oct 03 '19 at 15:58
  • it will work until it won't and at that moment you just bypass firestore capabilities, you need another solution, just for the specific case I've tested this patter with 500K and work without problem – andresmijares Oct 03 '19 at 16:07