0

I'm using MongoDB Realm and I would like to be able to delete duplicate items - could you advise on how I can do it?

I tried using dropDups or .distinct() but still didn't manage to make the code work.

  const {jobsPerPage = 100, page = 0} = payload.query;

  let query = {};
  if (payload.query.jobPosition) {
    query = { $text: { $search: payload.query.jobPosition } }
  } else if (payload.query.location) {
    query = { "location": { $eq: payload.query.location } }
  } else if (payload.query.companyName) {
    query = { $text: { $search: payload.query.companyName } }
  } else if (payload.query.jobDescription) {
    query = { $text: { $search: payload.query.jobDescription } }
  }
    
  const collection = context.services.get("mongodb-atlas").db("sample_jobs").collection("glassdoordbs").distinct();
  let jobsList = await collection.find(query).skip(page*jobsPerPage).limit(jobsPerPage).toArray()
  let allResults = await collection.count(query).then(num => num.toString())

  
  jobsList.forEach(job => {
    job._id = job._id.toString();
  });

  const responseData = {
    jobs: jobsList,
    page: page.toString(),
    filters: {},
    entries_per_page: jobsPerPage.toString(),
    total_results: await collection.count(query).then(num => num.toString()),
    totalPages: await Math.ceil(allResults / jobsPerPage).toString()

  };
  
  return responseData;
};```
Christopher
  • 57
  • 1
  • 5
  • What makes a document dup? there's no real other way except iterating over the collection and delete the dups "manually". – Tom Slabbaert Jul 09 '21 at 07:55
  • When I'm web-scraping (with puppeteer) and an element in an object is missing the scraper returns the previous item again instead of ignoring it and moving to the next. Here I added the code https://stackoverflow.com/questions/68306864/puppeteer-returns-the-most-recent-object-multiple-times – Christopher Jul 09 '21 at 07:58

0 Answers0