underscore/lodash unique by multiple properties

Question

I have an array of objects with duplicates and I'm trying to get a unique listing, where uniqueness is defined by a subset of the properties of the object. For example,

{a:"1",b:"1",c:"2"}

And I want to ignore c in the uniqueness comparison.

I can do something like

_.uniq(myArray,function(element) { return element.a + "_" + element+b});

I was hoping I could do

_.uniq(myArray,function(element) { return {a:element.a, b:element.b} });

But that doesn't work. Is there something like that I can do, or do I need to create a comparable representation of the object if I'm comparing multiple properties?

Answer 1

Use Lodash's uniqWith method:

_.uniqWith(array, [comparator])

This method is like _.uniq except that it accepts comparator which is invoked to compare elements of array. The order of result values is determined by the order they occur in the array. The comparator is invoked with two arguments: (arrVal, othVal).

When the comparator returns true, the items are considered duplicates and only the first occurrence will be included in the new array.

---

Example:
I have a list of locations with latitude and longitude coordinates -- some of which are identical -- and I want to see the list of locations with unique coordinates:

const locations = [
  {
    name: "Office 1",
    latitude: -30,
    longitude: -30
  },
  {
    name: "Office 2",
    latitude: -30,
    longitude: 10
  },
  {
    name: "Office 3",
    latitude: -30,
    longitude: 10
  }
];

const uniqueLocations = _.uniqWith(
  locations,
  (locationA, locationB) =>
    locationA.latitude === locationB.latitude &&
    locationA.longitude === locationB.longitude
);

// Result has Office 1 and Office 2

Answer 2

There doesn't seem to be a straightforward way to do this, unfortunately. Short of writing your own function for this, you'll need to return something that can be directly compared for equality (as in your first example).

One method would be to just .join() the properties you need:

_.uniqBy(myArray, function(elem) { return [elem.a, elem.b].join(); });

Alternatively, you can use _.pick or _.omit to remove whatever you don't need. From there, you could use _.values with a .join(), or even just JSON.stringify:

_.uniqBy(myArray, function(elem) {
    return JSON.stringify(_.pick(elem, ['a', 'b']));
});

Keep in mind that objects are not deterministic as far as property order goes, so you may want to just stick to the explicit array approach.

P.S. Replace uniqBy with uniq for Lodash < 4

Answer 3

Here there's the correct answer

javascript - lodash - create a unique list based on multiple attributes.

FYI var result = _.uniqBy(list, v => [v.id, v.sequence].join());

Answer 4

late to the party but I found this in lodash docs.

var objects = [{ 'x': 1, 'y': 2 }, { 'x': 2, 'y': 1 }, { 'x': 1, 'y': 2 }];
 
_.uniqWith(objects, _.isEqual);
// => [{ 'x': 1, 'y': 2 }, { 'x': 2, 'y': 1 }]

Answer 5

I do think that the join() approach is still the simplest. Despite concerns raised in the previous solution, I think choosing the right separator is the key to avoiding the identified pitfalls (with different value sets returning the same joined value). Keep in mind, the separator need not be a single character, it can be any string that you are confident will not occur naturally in the data itself. I do this all the time and am fond of using '~!$~' as my separator. It can also include special characters like \t\r\n etc.

If the data contained is truly that unpredictable, perhaps the max length is known and you could simply pad each element to its max length before joining.

Answer 6

There is a hint in @voithos and @Danail combined answer. How I solved this was to add a unique key on the objects in my array.

Starting Sample Data

const animalArray = [
  { a: 4, b: 'cat', d: 'generic' },
  { a: 5, b: 'cat', d: 'generic' },
  { a: 4, b: 'dog', d: 'generic' },
  { a: 4, b: 'cat', d: 'generic' },
];

In the example above, I want the array to be unique by a and b but right now I have two objects that have a: 4 and b: 'cat'. By combining a + b into a string I can get a unique key to check by.

{ a: 4, b: 'cat', d: 'generic', id: `${a}-${b}` }. // id is now '4-cat'

Note: You obviously need to map over the data or do this during creation of the object as you cannot reference properties of an object within the same object.

Now the comparison is simple...

_.uniqBy(animalArray, 'id');

The resulting array will be length of 3 it will have removed the last duplicate.