Creating an array of custom objects from nested data structure

Question

I'm working on a project that requires me to massage some API data (shown in the snippet below as 'apiData'). The data structure I ultimately need for the charting library I'm using (Recharts) is this:

[ 
 { date: '2018-04-24', TSLA: 283.37, AAPL: 250.01 },
 { date: '2018-04-25', AAPL: 320.34 } 
]

I've put together the function below and it works well enough, but I'm having trouble getting all the data to show up, even if there's no match between dates. In the below example, you'll notice that the object for date "2018-04-23" in the apiData is excluded. Ideally, the final ds would look like this:

[ 
 { date: '2018-04-23', TSLA: 285.12 }
 { date: '2018-04-24', TSLA: 283.37, AAPL: 250.01 },
 { date: '2018-04-25', AAPL: 320.34 } 
]

Also, there's probably a more performant way to do this, but I've been hacking away for a while and not seeing a better solution at the moment. E.g. the forEach isn't ideal as the data set grows, which it will when I need to plot long time periods of data.

So my questions are: 1) How can I make sure objects that match in date are combined while objects that don't are still included and 2) what's a more performant way to do this operation?

If anyone has any input or critique of my approach and how I can improve it, I'd greatly appreciate it.

Here's a link to a repl if it's more convenient then the code snippet below.

formatChartData = (data) => {
  const chartData = data
    .reduce((arr, stock) => {
      const stockArr = stock.chart.map((item) => {
        let chartObj = {};

        chartObj.date = item.date;
        chartObj[stock.quote.symbol] = item.close;

        if (arr.length > 0) {
          arr.forEach((arrItem) => {
            if (arrItem.date === item.date) {
              chartObj = { ...arrItem, ...chartObj };
            }
          });
        }
        return chartObj;
      });

      return stockArr;
    }, []);

  console.log(chartData)
}

const apiData = [
  {
    chart: [
      {
        date: "2018-04-23",
        open: 291.29,
        close: 285.12,
      },
      {
        date: "2018-04-24",
        open: 291.29,
        close: 283.37,
      },
    ],
    news: [],
    quote: {
      symbol: "TSLA"
    },
  },
  {
    chart: [
      {
        date: "2018-04-24",
        open: 200.29,
        close: 250.01,
      },
      {
        date: "2018-04-25",
        open: 290.20,
        close: 320.34,
      },
    ],
    news: [],
    quote: {
      symbol: "AAPL"
    },
  }
]

formatChartData(apiData)

EDIT: I ended up using charlietfl's solution with an inner forEach as I found this easier to read than using two reduce methods. The final function looks like this:

 const chartData = data
  .reduce((map, stock) => {
    stock.chart.forEach((chart) => {
      const chartObj = map.get(chart.date) || { date: chart.date };
      chartObj[stock.quote.symbol] = chart.close;
      map.set(chart.date, chartObj);
    });
    return map;
  }, new Map());`

Is there a reason you are using a reduce to loop through the array items? — C. Johnson, May 26 '18 at 23:36
I figured since I needed to reduce my raw data structure down to something smaller, using reduce might get me there. Though it does, I'm wondering if splitting up the responsibilities might be easier/clearer. — Adam Bohannon, May 27 '18 at 03:51

charlietfl · Accepted Answer · 2018-05-27T01:23:56.380

2

A cleaner way than having to loop through the accumulated new array each time to look for a date is to use one master object with dates as keys

Following I use reduce() to return a Map (could be an object literal also) using dates as keys and then convert the Map values iterable to array to get the final results

const dateMap = apiData.reduce((map,stock)=>{ 
   return stock.chart.reduce((_, chartItem)=>{
      // get stored object for this date, or create new object
      const dayObj = map.get(chartItem.date) || {date: chartItem.date};
      dayObj[stock.quote.symbol] = chartItem.close;
      // store the object in map again using date as key
      return map.set(chartItem.date, dayObj);
   },map);  
}, new Map)

const res = [...dateMap.values()];

console.log(res)

.as-console-wrapper {max-height: 100%!important;}

<script>
const apiData = [
  {
    chart: [
      {
        date: "2018-04-23",
        open: 291.29,
        close: 285.12,
      },
      {
        date: "2018-04-24",
        open: 291.29,
        close: 283.37,
      },
    ],
    news: [],
    quote: {
      symbol: "TSLA"
    },
  },
  {
    chart: [
      {
        date: "2018-04-24",
        open: 200.29,
        close: 250.01,
      },
      {
        date: "2018-04-25",
        open: 290.20,
        close: 320.34,
      },
    ],
    news: [],
    quote: {
      symbol: "AAPL"
    },
  }
]


</script>

edited May 27 '18 at 01:23

answered May 26 '18 at 23:46

charlietfl

170,828
13
121
150

This is great! Thanks for taking the time to share this. – Adam Bohannon May 27 '18 at 04:46
Sorry, one more question: out of curiosity, could you explain exactly how the second reduce is working? I see you're passing as the accumulator to the inner reduce the map that's initialized in the outer reduce... and then that inner map is being combined with the outer map? – Adam Bohannon May 27 '18 at 04:57
It is all one Map and as you point out it starts in outer and gets passed into inner. So instead of returning accumulator in outer... am simply returning same thing as resultant of inner – charlietfl May 27 '18 at 06:03
I actually did this at first just using a `forEach` loop inside outer reduce. Switching to inner reduce removed one line. Could use any loop inside outer reduce and return the map(accumulator) – charlietfl May 27 '18 at 06:07
I ended up using the forEach inside the outer reduce. I found this easier to read than using an inner reduce. Thanks again. – Adam Bohannon May 27 '18 at 20:55

score 0 · Answer 2 · answered May 26 '18 at 23:50

Just correcting your code only, else reduce should be used instead of map, for charts also.

formatChartData = (data) => {
  const chartData = data
    .reduce((arr, stock) => {
      const stockArr = stock.chart.map((item) => {
        let chartObj = {};

        chartObj.date = item.date;
        chartObj[stock.quote.symbol] = item.close;

        if (arr.length > 0) {
          arr.forEach((arrItem, i) => {
            if (arrItem.date === item.date) {
              chartObj = { ...arrItem, ...chartObj };
              delete(arr[i]);
            }
          });
        }
        return chartObj;
      });

      return [...arr, ...stockArr].filter(e=>!!e); //to remove undefined elements left by delete above.
    }, []);

  console.log(chartData)
}

const apiData = [
  {
    chart: [
      {
        date: "2018-04-23",
        open: 291.29,
        close: 285.12,
      },
      {
        date: "2018-04-24",
        open: 291.29,
        close: 283.37,
      },
    ],
    news: [],
    quote: {
      symbol: "TSLA"
    },
  },
  {
    chart: [
      {
        date: "2018-04-24",
        open: 200.29,
        close: 250.01,
      },
      {
        date: "2018-04-25",
        open: 290.20,
        close: 320.34,
      },
    ],
    news: [],
    quote: {
      symbol: "AAPL"
    },
  }
]

formatChartData(apiData)

score 0 · Answer 3 · answered May 26 '18 at 23:57

The problem is that your reduce function is running for each item in the data array. When it runs on the first item in the data array it returns:

[
    { date: '2018-04-23', TSLA: 285.12 },
    { date: '2018-04-24', TSLA: 283.37 }
]

When it runs on the second item in the array it returns this:

[
    { date: '2018-04-24', TSLA: 283.37, AAPL: 250.01 },
    { date: '2018-04-25', AAPL: 320.34 } 
]

This is because when the reduce runs on the last array item it is returning the result from that item. You are only merging items from the accumulator variable "arr" if their date is in the current array item as well. Since 2018-04-23 is in the first but not the second it is not being added. I have added two things to your code. First if the current date being looped on is in the accumulator variable "arr" I delete it from "arr" after it is merged in. The second change is after each .reduce loop there is still some dates left in "arr" that aren't in the current "stockArr". To deal with this I merge both "arr" and "stockArr" which will give you what you are looking for.

formatChartData = (data) => {
  const chartData = data
    .reduce((arr, stock) => {
      const stockArr = stock.chart.map((item) => {
        let chartObj = {};

        chartObj.date = item.date;
        chartObj[stock.quote.symbol] = item.close;
        
        if (arr.length > 0) {
          arr.forEach((arrItem, index) => {
            if (arrItem.date === item.date) {
              chartObj = { ...arrItem, ...chartObj };
              arr.splice(index, 1);
            }
          });
        }
        return chartObj;
      });
      return [...arr, ...stockArr];
    }, []);

  console.log(chartData)
}

const data = [
  {
    chart: [
      {
        date: "2018-04-23",
        open: 291.29,
        close: 285.12,
      },
      {
        date: "2018-04-24",
        open: 291.29,
        close: 283.37,
      },
    ],
    news: [],
    quote: {
      symbol: "TSLA"
    },
  },
  {
    chart: [
      {
        date: "2018-04-24",
        open: 200.29,
        close: 250.01,
      },
      {
        date: "2018-04-25",
        open: 290.20,
        close: 320.34,
      },
    ],
    news: [],
    quote: {
      symbol: "AAPL"
    },
  }
]

formatChartData(data)

Creating an array of custom objects from nested data structure

3 Answers3