How should I avoid duplicate data and API calls in a flat vuex structure?

Question

I am currently building a vue/vuex/firestore/vuexFire project and I ran into the following issue.

Say I want to display a list of messages where each message is associated with a user. A user can submit multiple messages, which will all be under their name. In vuex, a flat representation of this might look like:

User state module

state: () => ({
  users: [
   {
    name: 'foo',
    id: 'bar'
   }
   //...
  ]
})

Messages state module

state: () => ({
  messages: [
   {
    message: 'blah blah',
    authorId: 'bar'
   }
   //...
  ]
})

On the server (I am using Firebase's Firestore) the structure is essentially the same. So, I fetch all the messages, then follow each authorId like so:

// action in the vuex messages module
{
 fetchMessages: async ({ commit, dispatch }) => {
  
  const snapshot = await firestore.collection('messages').get();
  snapshot.forEach((doc) => {
   
   // commit the document
   commit('addMessage', doc.data())
   
   // fetch the associated user
   dispatch('fetchUser', doc.data().authorId)
  })
 },

 fetchUser: async ({commit}, id) => {
  const user = await firestore.collection('users').doc(id).get();
  commit('addUser', user.data())
 }

}

The issue I am having is that, if there is a user who has created multiple messages, they will be added to the users array over and over. Additionally, fetchUser() is called unnecessarily, over and over. So far, my attempts at solving this have failed or were inelegant:

I tried to check if the user is already present in the users array before calling fetchUser(). However, this doesn't work, because fetchUser is an async operation. It could be in the process of fetching, so checking to see if the user is already in the array usually tests false.
I tried to check if the user is already present in the addUser mutation. While this does prevent duplicates, it does not prevent the unnecessary fetch of the user. Also, the addUser mutation has to check that the user is not a duplicate on every call, which could be expensive with larger arrays.
One approach that did work was creating a fetchJobs object in the users state. When fetchUser() was called, I added the id of the user to the object. Then, before fetching additional users, I would check against this object to make sure that I was not fetching a duplicate. The reason this worked, as opposed to directly checking the state, is that I committed the fetchJob at the beginning of the action, so it was not asynchronous. That way, other components and state modules could access it. However, this does not work if the user object needs to be updated at any point, as it will still remain in the fetchJobs object.

I should also mention that I am using a small helper library called VuexFire, which manages bindings for Firestore collectionReference.onSnapshot(). This has introduced another limitation, which is that I cannot directly run an action when a document is added, updated, or deleted. This is because VuexFire handles all of these changes automatically, and doesn't support hooking into these events. Instead, I have to loop through all of the existing messages and call fetchUser(). I didn't include this in the above code examples because it is not much different from snapshot.forEach(). If need be, I can also ditch this library.

So, what is the best way to approach this problem?

Have you tried to get messages linked to each user? Is this possible using Firestore? It seems expensive to fetch all messages and then loop through them yourself. Every database I have worked with has the ability to get information based off an id (in this case, message information based off of authorId) and databases tend to do it more efficiently than we do it with logic. — Tony, Aug 05 '20 at 09:23

score 2 · Accepted Answer · answered Aug 05 '20 at 10:34

From everything you have typed, it seems like preventing the fetchUser action from being called unnecessarily is paramount so I will focus on the one attempt you have tried, to prevent that from happening

I tried to check if the user is already present in the users array before calling fetchUser(). However, this doesn't work, because fetchUser is an async operation. It could be in the process of fetching, so checking to see if the user is already in the array usually tests false

How about this?

{
 fetchMessages: async ({ commit, dispatch }) => {
  
  const snapshot = await firestore.collection('messages').get();
  snapshot.forEach((doc) => {
   
   // commit the document
   commit('addMessage', doc.data())
   
   const userIndexInTheUsersStoreArray = store.state.users.findIndex(user => user.id === doc.data().authorId)
   
   const isUserInTheUsersStoreArray = userIndexInTheUsersStoreArray > -1
   
   if (!userInTheUsersStoreArray) {
    dispatch('fetchUser', doc.data().authorId)
   }
  })
 },

This is much smarter than my fetchJobs solution - Thanks! – Sevy Aug 05 '20 at 15:53 — Sevy, Aug 05 '20 at 15:53

How should I avoid duplicate data and API calls in a flat vuex structure?

1 Answers1