How to skip a duplicate object using a property value within reduce function in javascript?

Question

I have an array of objects like this:

// other properties of the object omitted for brevity
// this array can potentially contain upto 50 objects like this
var myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}]

I am trying to apply the reduce function to create a new object like this:

  var newArray = myArray.reduce(function(acc, current, index){
  var newObj = {};
  newObj['strata']  = 'kit';
  newObj['href'] = current['url'];
  acc.push(newObj);
  return acc;  
}, [])

But I do not want to include duplicate objects ('duplicate' tested using 'url' property of the object). How can i modify my reduce function to skip these kinds of objects and produce an

[{strata: kit, href: 'http://linkA'}, {strata: kit, href: 'http://linkB'}]

Edit: Sorry, i forgot to mention that this is legacy code. I cannot use features like 'Set' and 'some'

Possible duplicate of [Remove duplicate values from an array of objects in javascript](https://stackoverflow.com/questions/45439961/remove-duplicate-values-from-an-array-of-objects-in-javascript) — prasanth, May 17 '19 at 09:31
Note that when you're unconditionally pushing to a new array you've passed in as the accumulator in a `reduce` (as above), you're using the wrong tool; use `map` instead. (But if it's conditional, `map` wouldn't be appropriate and a simple loop is best.) — T.J. Crowder, May 17 '19 at 09:31

CertainPerformance · Accepted Answer · 2019-05-17T10:02:23.973

1

You can check whether the url exists in any of the objects in the accumulator first:

var myArray = [{
  url: 'http://linkA'
}, {
  url: 'http://linkB'
}, {
  url: 'http://linkA'
}];

var newArray = myArray.reduce(function(acc, { url }, index) {
  if (acc.some(obj => obj.href === url)) {
    return acc;
  }
  acc.push({
    strata: 'kit',
    href: url
  });
  return acc;
}, [])

console.log(newArray);

For O(N) complexity instead of O(N^2), add each URL to an outer Set (whose lookup is O(1), rather than O(N) for Array.prototype.some):

var myArray = [{
  url: 'http://linkA'
}, {
  url: 'http://linkB'
}, {
  url: 'http://linkA'
}];

const urls = new Set();
var newArray = myArray.reduce(function(acc, { url }, index) {
  if (urls.has(url)) {
    return acc;
  }
  urls.add(url);
  acc.push({
    strata: 'kit',
    href: url
  });
  return acc;
}, [])

console.log(newArray);

Or, in ES5, use an object instead of a Set:

var myArray = [{
  url: 'http://linkA'
}, {
  url: 'http://linkB'
}, {
  url: 'http://linkA'
}];

var urls = {};
var newArray = myArray.reduce(function(acc, obj, index) {
  var url = obj.url;
  if (urls[url]) {
    return acc;
  }
  urls[url] = true;
  acc.push({
    strata: 'kit',
    href: url
  });
  return acc;
}, [])

console.log(newArray);

edited May 17 '19 at 10:02

answered May 17 '19 at 09:29

CertainPerformance

356,069
52
309
320

Can I ask why use `Object.values(acc).some` rather than `acc.some`? – Maheer Ali May 17 '19 at 09:32
Oops, it's already an array, you're right. I'm used to the `.reduce`'s accumulator, when not a primitive, being an object – CertainPerformance May 17 '19 at 09:40
Can you please suggest how can i replace acc.some as this is legacy code ? – Adhyatmik May 17 '19 at 09:43
@Adhyatmik Best to use Babel and [polyfills](https://polyfill.io/v3/) so that you can write in the latest and greatest version of the language - but if not, see edit, use an object instead of a Set, like in the bottom snippet – CertainPerformance May 17 '19 at 09:46
Should be: `if (urls[url]) { }` ? – adiga May 17 '19 at 10:01

Maheer Ali · Answer 2 · 2019-05-17T09:36:45.077

You can check for duplicates using some() before push()

var myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}]

var newArray = myArray.reduce(function(acc, current, index){
  var newObj = {};
  newObj['strata']  = 'kit';
  newObj['href'] = current['url'];
  if(!acc.some(x => x.href === current.url)) acc.push(newObj);
  return acc;  
}, [])

console.log(newArray)

A shorter version of code will be

var myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}]

const res=  myArray.reduce((ac,a) => 
                  (!ac.some(x => x.href === a.url) && 
                  ac.push({strata:'kit',href:a.url}),
                  ac),
             []);

console.log(res)

score 1 · Answer 3 · answered May 17 '19 at 09:30

Use some:

var myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}];

var newArray = myArray.reduce(function(acc, { url }, index) {
  if (Object.values(acc).some(({ href }) => href === url)) {
    return acc;
  }
  acc.push({
    strata: 'kit',
    href: url
  });
  return acc;
}, [])

console.log(newArray);

adiga · Answer 4 · 2019-05-17T09:53:26.803

You could get all the unique urls using Set first. Then create the objects using map and Shorthand property names:

const myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}],
      uniqueUrls = new Set(myArray.map(a => a.url)),
      strata = "kit",
      output = [...uniqueUrls].map(href => ({ strata, href }));

console.log(output)

If you can't use ES2015+ features, you can have an object as accumulator and each unique url as key. This way you'll get only one url in the accumulator. Then loop through the object and get the values of the merged object

var myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}];

var merged = myArray.reduce(function(acc, current) {
  acc[current.url] = current;
  return acc;
}, {})

var output = [];

for(var key in merged) {
  output.push(merged[key])
}

console.log(output)

This is what the merged object will look like:

{
  "http://linkA": {
    "url": "http://linkA"
  },
  "http://linkB": {
    "url": "http://linkB"
  }
}

Francis Leigh · Answer 5 · 2019-05-17T10:11:22.567

create a Set that accepts a mapped version of you array with only the urls in - The Set wil de-dupe your array and then you can create a new array from the Set with the strata property added to it.

const myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}]
const s = new Set(myArray.map(({ url }) => url))

const newArray = [...s].map(url => ({
  url,
  strata: 'kit'
}))
console.log(newArray)

If you want your final Array of Objects to be more dynamic. Create a function that takes the parameter you want to de-dupe by and the params you want to insert into the finalized Object.

const DE_DUPE_INSERT = (array, flag, insert) => {
  const s = new Set(array.map(item => item[flag]))
  return [...s].map(property => ({
    [flag]: property,
    ...insert
  }))
}
const ARR = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}]
const de_duped = DE_DUPE_INSERT(ARR, 'url', {strata: 'kit', foo: 'bar'})

console.log(de_duped)

< es6

var DE_DUPE_INSERT = function (array, flag, insert) {
  var de_duped = []
  array.forEach(function (item) {
    if (de_duped.indexOf(item[flag]) === -1) de_duped.push(item[flag])
  })
  
  return de_duped.map(function (property) { 
    var obj = {}
    obj[flag] = property
    Object.keys(insert).forEach(function (key) {
      obj[key] = insert[key]
    })
    return obj
  })
}
var ARR = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}]
var de_duped = DE_DUPE_INSERT(ARR, 'url', {strata: 'kit', foo: 'bar'})

console.log(de_duped)

T.J. Crowder · Answer 6 · 2019-05-17T09:39:22.680

I'd probably

Use a Set (or an object) to keep track of URLs I'd already seen, and
Use an object initializer rather than three statements to create the new object, and
Use a simple loop rather than reduce

But here's a version still using reduce:

var myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}]
var known = new Set();
var newArray = myArray.reduce(function(acc, current, index){
  if (!known.has(current.url)) {
    acc.push({
      strata: 'kit',
      href: current.url
    });
    known.add(current.url);
  }
  return acc;  
}, [])
console.log(newArray);

Here's the to-my-mind simpler version using a loop:

var myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}]
var known = new Set();
var newArray = [];
for (const current of myArray) {
  if (!known.has(current.url)) {
    newArray.push({
      strata: 'kit',
      href: current.url
    });
    known.add(current.url);
  }
}
console.log(newArray);

If you can't use ES2015+ features like Set and for-of, you can use an object to keep track of URLs and a simple for loop:

var myArray = [{url: 'http://linkA'}, {url: 'http://linkB'}, {url: 'http://linkA'}]
var known = Object.create(null); // An object with no prototype
var newArray = [];
for (var index = 0; index < myArray.length; ++index) {
  var current = myArray[index];
  if (!known[current.url]) {
    newArray.push({
      strata: 'kit',
      href: current.url
    });
    known[current.url] = true;
  }
}
console.log(newArray);

Sorry, i forgot to mention. This is legacy code pre-ECMAScript 6. I dont think Set and some will work. — Adhyatmik, May 17 '19 at 09:38
@Adhyatmik - Ah, then see the ES5 solution I added a moment ago at the end. :-) — T.J. Crowder, May 17 '19 at 09:38

How to skip a duplicate object using a property value within reduce function in javascript?

6 Answers6