1

I am building an internal app that displays data in an HTML table. The API returns me an array of up to 20K objects which are stored in data() and the table is populated by each cell calling a method to find the correct object. The object is located via a 3-part key (2 for the row and 1 for the column). Initial rendering performance on my laptop (i7 7thGen and 8GB ram) is acceptable [about 1 minute] for initial render (excel app it is replacing takes almost 3), however an update to a single cell (object in the array) triggers the change detection and an update takes another minute. Sometimes the user will want to update a single cell, sometimes a row and sometimes propagate a change on all rows(or some selected rows) for a single column. Is there a recommended strategy for performance enhancement. I was thinking of restructuring the data that comes back into an object per row (2-part key) with a collection of objects for the column (1-part key). This would mean that the method call will only have to iterate over 2500 'row' objects and then over 8 'column' objects which feels logically like it should be faster but will now consume more memory. I need to be able to identify which objects have changed at a table cell level as I need to write the changes back to the database. Should I discard the original API results and re-hydrate on save or would an option be to use Object.freeze to prevent reactivity on the original array and write changes to a separate array which then takes precedence if a record exists.

[
  {
     key1:value,
     key2:value,
     key3:valueA,
     displayValue:value
  },
  {
     key1:value,
     key2:value,
     key3:valueB,
     displayValue:value2
  },
  {
     key1:value,
     key2:value,
     key3:valueC,
     displayValue:value3
  }...
  {
     key1:value2,
     key2:value2,
     key3:valueA,
     displayValue:value
  },
  {
     key1:value2,
     key2:value2,
     key3:valueB,
     displayValue:value2
  },
  {
     key1:value2,
     key2:value2,
     key3:valueC,
     displayValue:value3
  }...
]

becomes

[
 {
   key1:value,
   key2:value,
   key3collection:[
     {key3:valueA, displayvalue:value},
     {key3:valueB, displayvalue:value2},
     {key3:valueC, displayvalue:value3}
   ]
 },
 {
   key1:value2,
   key2:value2,
   key3collection:[
     {key3:valueA, displayvalue:value},
     {key3:valueB, displayvalue:value2},
     {key3:valueC, displayvalue:value3}
   ]
 },
]
mare96
  • 3,749
  • 1
  • 16
  • 28
Aaron Reese
  • 544
  • 6
  • 18
  • "table is populated by each cell calling a method to find the correct object" ...this sounds expensive. You should definitely transform API result to something that could be iterated over. How does your "save" API looks like ? Can be keys (1/2/3) changed by user or only `displayvalue` ? – Michal Levý Nov 29 '19 at 14:24
  • Best practice is to normalize your data so that it doesn't require lookups to create relational views, so this is where you'll want to start. Have a look at LinusBorg's (Vue leader) reply on this forum post: https://forum.vuejs.org/t/vuex-best-practices-for-complex-objects/10143 – Dan Nov 29 '19 at 14:54
  • Data structure is definitely a problem. But it looks to me as he's doing lookups even for rendering which sounds really bad. Maybe show us the important parts of the component itself ? – Michal Levý Nov 29 '19 at 15:02
  • Thanks Guys. I come from an RDBMS background so I think in tables rather than documents. Looking at the post from Linusborg I think I am almost there conceptually. Instead of an [ ] of { } it needs to be an { } with unknown props of type { }. I can't use vuex or webpack and need to keep the code readable. Do you think it might make more sense to do the conversion in the middleware (PHP). I am using MSSQL2012 so I can even do FOR XML and then have either PHP or js convert to json. Thoughts? – Aaron Reese Dec 01 '19 at 10:55
  • So is each object in your question actually representation of single row in some MS SQL table ? – Michal Levý Dec 01 '19 at 12:12
  • unfortunately not. if we are talkng 3NF then I have priceGroup, market, priceBreak, market_x_priceBreak (aka priceList), and priceGroup_x_priceList which is temorarlly controlled with contiguous valid_from,valid_to dates. I also have some tables used to control the calculation rules for future dates (most prices are done on a formula but some are manually overridden) The nested levels are an array of priceGroup_x_priceList records but picking up the active, future-set, future-calculated and the rules used for the calculation. I am trying to keep the example simple to make it generic. – Aaron Reese Dec 01 '19 at 21:41

1 Answers1

0

What I would do is to convert API response (directly on server if you control API or on the client) to something like this:

{
  columns: ['ValueA', 'ValueB', 'ValueC']  // same for every row, right ?
  rows: [
    {
      key1:value,
      key2:value,
      values:[ 'value', 'value2', 'value3' ]
    }, {
      key1:value2,
      key2:value2,
      values:[ 'value', 'value2', 'value3' ]
    }, ....
]

I'm working with an assumption that every row has same set of columns so why to keep duplicity in the data.

That way you can render the table by following template (pseudo code) without any additional data lookups:

<table>
  <thead>
    <tr>
      <th v-for="column in data.columns">{{ column }}</th>
    </tr>
  </thead>
  <tbody>
    <tr v-for="(row, rowIndex) in data.rows">
      <td v-for="(rowItem, columnIndex) in row.values">{{ rowItem }}</td>
    </tr>
  </tbody>
</table>

For change tracking (so you can later send it back to API) just add simple array alongside values like isDirty: [ fasle, false, false ] and change it to true every time you update some cell. When posting just iterate over data structure and recreate updated objects in the format your API expects it....

Michal Levý
  • 33,064
  • 4
  • 68
  • 86
  • I have an array of columns to do the column iteration but the format is more complex because it is actually an array of markets, each of which has an array of price lists. Unfortunately the values array is an array of objects rather than primitives and although in 99.9% of cases I will have an object for each column I cannot guarantee it and I can't trust the order of data from the API (defensive programming...) How does the change tracking array help? Vue will still try to render the whole table. I will need to track dirty records for the update API though. – Aaron Reese Dec 01 '19 at 21:50
  • Change tracking is exactly for that - so you know which cell has changed and need to be posted back to server. Vue will __always__ render whole table (to virtual DOM). That's why iterating over data will always be __much__ faster then looking up correct object in huge array for every cell... – Michal Levý Dec 01 '19 at 22:04
  • Off topic, but is there a way to 'suspend' the change tracking. for example if I need to change values[3][2].active_date for each element in the rows[] array (or each property in the rows{} object), it would make sense to complete all the data updates before the re-render occurs – Aaron Reese Dec 01 '19 at 22:13
  • Vue never triggers re-render in the middle of your code working. [That's something not possible in JavaScript](https://stackoverflow.com/a/59095359/381282) – Michal Levý Dec 01 '19 at 22:16
  • `Unfortunately the values array is an array of objects rather than primitives` - not problem at all, just put an object in place of primitive value. `in 99.9% of cases I will have an object for each column I cannot guarantee` - not problem, just put `null` there and handle the case while rendering. `I can't trust the order of data from the API` - ok, sort it then. Point is, do all this stuff __only once__ after you receive the data so rendering can be just iteration... – Michal Levý Dec 01 '19 at 22:28