1

I have an array of objects, I would like to remove duplicates. My array has a common field name that I would like to use for deduplication.

I am trying to convert the array to a map and then from map back to array but map conversions gives me an error duplicate field name: "a":

local arr = [ 
    { "name": "a", "value": 1234},
    { "name": "b", "value": 555},
    { "name": "c", "value": 0}, 
    { "name": "a", "value": 1234} 
];
local map = { [x.name] : x  for x in arr };

Desired output: 

[ 
      { "name": "a", "value": 1234},
      { "name": "b", "value": 555}, 
      { "name": "c", "value": 0} 
]
Aleš
  • 8,896
  • 8
  • 62
  • 107

2 Answers2

2

As @seh pointed out in ksonnet channel, the latest jsonnet release now allows to use std.set() on objects.

 local arr = [
    { name: "a", value: 1234 },
    { name: "b", value: 555 },
    { name: "c", value: 0 },
    { name: "a", value: 1234 },
  ];
 std.set(arr, function(o) o.name)

The std.set() header is documented in jsonnet's std lib implementation.

Aleš
  • 8,896
  • 8
  • 62
  • 107
  • 1
    beware that above (using `std.set()`) sets result to `{ name: "a", value: 4321 }` rather than `1234`. – jjo Dec 06 '18 at 12:17
1

Ignoring original sort order

You can implement it by replacing the comprehension with std.foldl(), do note though the ordering issue:

local arr = [
  { name: "a", value: 4321 },
  { name: "b", value: 555 },
  { name: "c", value: 0 },
  { name: "a", value: 1234 },
];

// Use foldl to iterate from array, can't use comprehension because of dup fields
local map = std.foldl(function(x, y) x { [y.name]: y }, arr, {});

// Re-convert map to array, note that it'll not respect original order
// but fields' (ie 'name' key)
[ map[x] for x in std.objectFields(map)]

Keeping original sort order

If you need to keep original sort order in output array, you can then add an _idx field to use in a final sort():

local arr = [
  { name: "a", value: 4321 },
  { name: "b", value: 555 },
  { name: "c", value: 0 },
  { name: "a", value: 1234 },
];

// Overload array elements with there index (`_idx` field)
local idxArray = std.mapWithIndex(function(i, x) x { _idx:: i }, arr);

// Use foldl to iterate from array, can't use comprehension because of dup fields
local map = std.foldl(function(x, y) x { [y.name]: y }, idxArray, {});

// Re-convert map to array, it'll keep original order via added _idx field
std.sort([map[x] for x in std.objectFields(map)], function(e) e._idx)
jjo
  • 2,595
  • 1
  • 8
  • 16
  • Thanks, I ended up using this `std.foldl()` approach in the end. The latest jsonnet release also allows to use `std.set(arr, keyF=id)` too. – Aleš Nov 28 '18 at 21:41