0

I'm currently re-structuring one of my Firestore documents because of too many read operations.

With the following structure, a document size will be around 90,000 bytes. Is this OK or should I structure my document differently?

If you have any suggestions, please let me know :)

{
  name: "Acme",
  date_added: "November 5, 2020 at 3:12:16 PM UTC+1",
  id: "f9hh9tvkEd9dgjf3r",
  bg_color: "333333",
  sections: [ // Max 8 sections like the one below
    {
      section_type: "color",
      label: "user defined label",
      description: "some description",
      grid: 4,
      items: [
        {
          cmyk: { c: "0", m: "39", y: "100", k: "20" },
          hex: "C20D19",
          rgb: { r: "194", g: "13", b: "25" },
          name: "Rouge",
          date_added: "November 5, 2018 at 3:12:16 PM UTC+1"
        }
        // ... ~20 of these items
      ]
  ],
  users: [
    {
      name: "John Appleseed",
      email: "john@appleseed.com",
      id: "Kf3ghxS52fZ9G6Y1b8BD5pv6Cfn5",
      role: "member",
      date_added: "November 10, 2020 at 09:33:27 PM UTC+1"
    },
    // Let's say there's 300 users
  ]
}

Nicholas
  • 501
  • 2
  • 14

2 Answers2

2

With the following structure, a document size will be around 90 000 bytes. Is this OK?

As long as you stay within the maximum size that is allowed for a document which is 1 MiB (1,048,576 bytes), you can go ahead with your schema. Doesn't matter if your document has the size of 90,000 bytes (0.09 MiB) or the max of 1 Mib, your app will work and scale perfectly fine.

I'm currently re-structuring one of my Firestore documents because of too many read operations.

If you are storing in the documents large amounts of data and the documents should be updated by lots of users, there is another limitation that you need to take care of. So you are limited to 1 write per second on every document. So if you have a situation in which a lot of users al all trying to write/update data to the same documents all at once, you might start to see some of this writes to fail. So, if you reached that limitation, then you should reconsider your document structure by getting out the data within an array for example in a separate sub-collection.

or should I structure my document differently?

We usually structure the database according to the queries we want to perform. Without knowing that, it's hard to say which schema might be better. So if have hard times with that, please post another question with the actual database schema and the queries you need, so we can take a look.

Alex Mamo
  • 130,605
  • 17
  • 163
  • 193
  • Thanks, Alex! I guess I'm safe when it comes to document size. Regarding updating the document, this will usually be done by one user only and not often. Except when users adds the document, their uid will be added to the user array. And about the queries, the whole document, as shown above, will be queried at the same time. That's the problem I have with the existing structure, because each section has their own sub-collection. – Nichlas Wærnes Andersen Jan 08 '20 at 14:35
  • *I'm safe when it comes to document size*, Yes you are. *Regarding updating the document, this will usually be done by one user only and not often.* It's good then. *Each section has its own sub-collection.* I'm afraid I did not understand you since I'm not seeing any sub-collection in your document, only just properties of type String and an array. If you want some thought regarding your entire database schema, please post another fresh question, so I and other Firebase developers can help you. – Alex Mamo Jan 08 '20 at 15:00
  • @AlexMamo one other possible issue I'm seeing with document size is impact on latency compensation. This is an important tradeoff for me as I think about how much to store in 1 document vs many documents. From my testing, I'm seeing inconsistent results. Which should a bigger impact on latency compensation: query result size or document size? Full question and findings so far here: https://stackoverflow.com/questions/61254151/should-i-build-a-local-data-layer-app-state-to-maintain-state-in-a-react-native – learningAngular May 07 '20 at 14:16
  • @learningAngular Yes, that's a trade between those two options. According to your requirements, you should choose one, or the other. – Alex Mamo May 07 '20 at 14:25
1

Maximum size of a document in Firestore is 1MB. That being said you are safe but...

  • If that document is changed often, whole document will be downloaded by each user on every change which could increase your network egress and have some impact on your user's mobile data plans. Still from my experience it is much easier to reach reads limit than data one.
  • If you need users to be able to change that document you can not manage it through Firestore security rules but you would have to use Functions which also have a limit on number of runs.
  • I see a potential privacy issue. In your current setup all of the users will have access to personal data of other users (name, surname, email, etc). Consider maybe putting users in two separate documents. One public document that is accessible by all while you would have one private document if data is accessible and editable only by you. If users need to edit their own data than you would have to have for each user private and public document and one public index which would contain the data that should be seen by all. You would than have to update that public index using Functions.
radulle
  • 1,437
  • 13
  • 19
  • Great answer, thank you! To clearify my use-case a bit better: a document will mostly have writes to it when it's first set up by the owner, and after that mostly read operations. And good tip about user privacy, what I could is store the uid together with the role, and that should solve it I think? This is a desktop app only, so I don't think the data size will be a big problem? – Nichlas Wærnes Andersen Jan 08 '20 at 14:15
  • 1
    You do not have to store roles in Firestore unless they need to be seen by other users. You can set roles in Custom Claims (user token) [link](https://firebase.google.com/docs/auth/admin/custom-claims). Than when setting up Firestore Security Rules you do not have to read from DB which counts as a read to get the role but directly from the user's token which is free [link](https://www.youtube.com/watch?v=3hj_r_N0qMs) – radulle Jan 08 '20 at 14:35
  • Oh, I didn't know, that's great! But in my case, a user can have access to more documents like the one I posted, where their role can be something else ("admin" or "member") – Nichlas Wærnes Andersen Jan 08 '20 at 14:42
  • 1
    You can assign each user several roles. Check those two links I provided to get a better picture what can be done, you can save many reads. – radulle Jan 08 '20 at 14:50