Best way to filter for duplicates?

Question

I posted about this before trying to use a regular expression, but this time is a bit different.

So I have this list of 500 phone numbers. Here is a little sample of them:

It looks a bit different, this is on my mongoDB server, so its more like an array of objects with a unique uid... so for example:

[
    {
        _id: kasjf234lkj109euf9023u4n,
        field1: 8669894327,
    }
]

something like that right.

So basically, my webapp is using a csv file of numbers that you want to compare to your base numbers.

My first thought was using a for, while, for loop kind of thing but im not sure how well it'll work:

for(var i = 0; i < basenums.length; i++){
    while ( i >= 0 ){
        for(var j = 0; j < comparing.length; j++){
            if comparing[j] == basenums.field1[i]{
                push that number to a 'dupes' array
            }else{
                break or something?
            }
        }
    }
}

...this logic is starting to hurt my head...

I know theres an 'includes()' method, but I havn't really used it and when I tried it in this case, It gave me everything as false, even tho the list I was using to compare is just a copy of my 'basenums' list on my server.

What would be the more 'correct' way of doing this?

I'm trying to compare an array with an array. Not a value in its specific array to its own array.

So, rather than marking this as a duplicate, maybe you should re-read:

I have 2 arrays of numbers: [1, 2, 3, 4, 5, 6] and [1, 2, 3, 4, 5, 6] How do I take array a index 0 and compare it to array b every index, then take index 1 of array a and compare to every index in array b, so on and so forth until i > arrayA.length.

In the case of finding a match when comparing the arrays. What action should be taken? — Mario, Sep 04 '18 at 15:41

JanS · Accepted Answer · 2018-09-04T14:36:08.207

1

Don't try to do everything in one single step. Break your problem into smaller chunks:

Format mongo's response into a hash map that you can efficiently query (O(1))
Test each input number against this map

Step 1:

var hashmap = {}
basenums.forEach(function(basenum) {
  hashmap[basenum.field1] = true;
})

Step 2:

var dupes = [];
comparing.forEach(function(comparingNum) {
  if(hashmap[comparingNum]) {
    dupes.push(comparingNum);
  }
})

EDIT: I am not sure, whether you want to have your dupes as a set (unique array). If so, you could use an alternate Step 2:

var dupesMap = {};
comparing.forEach(function(comparingNum) {
  if(hashmap[comparingNum]) {
    dupesMap[comparingNum] = true;
  }
})
var dupes = Object.keys(dupesMap);

edited Sep 04 '18 at 14:36

answered Sep 04 '18 at 14:30

JanS

2,065
3
27
29

Hmm.. I'll look into this a bit..It seems like it would work, but honestly I'm barely understanding whats going on here.......What exactly is the object 'hashmap'..? each instance of basenum.field1? – Ext- Sep 04 '18 at 14:41
In JavaScript, any Object (`{}`) is a map - a key/value store. In this case, I called it hashmap. I could also have called it *kownNumberStore* or something like that. Next, I am building a map/Object like `{'8664665844': true, '8885444636': true}` so that I can ask that Object via `hashMap['8885444636']` wether there is an entry/value with that number. It doesn't do more - it just serves as a lookup tool. – JanS Sep 04 '18 at 14:47
just to be clear, I'll have to make the logic to search the array, yeah? because its just showing up as everything is true. lol – Ext- Sep 04 '18 at 15:19
This dosn't seem to work though I'm not sure why... hashmap creates and object with key value pairs of 'basenum': true just like it says to do. tho, it dosn't seem to be going into the second forEach loop.. i guess i'll make this separate methods and try again. – Ext- Sep 04 '18 at 15:34
yeah it seems to not be going into the next forEach loop for whatever reason.. – Ext- Sep 04 '18 at 15:41
for some reason, it would not loop into the second forEach loop because it wanted me to click a start button twice..IDK why. but ny way. this worked. tyvm for the help :) – Ext- Sep 04 '18 at 16:10

score 0 · Answer 2 · answered Sep 04 '18 at 14:33

0

You can push values into an array and create a new set with the array. Sets do not keep redundant elements in them. Try this;

let a = ["a","b","a","b","a","b","a","b","a","c"];
console.log(a);
let uniqueA = [...new Set(a)];
console.log(uniqueA);

By the way, for piece of code I got a help from this page: https://medium.com/front-end-hacking/getting-unique-values-in-javascript-arrays-17063080f836

answered Sep 04 '18 at 14:33

Ekrem Demirhan

204
3
11

This would work but I have arrayA and arrayB, and would rather keep them this way for the time being. – Ext- Sep 04 '18 at 14:36

Best way to filter for duplicates?

2 Answers2