How to remove duplicate records from NgFor Loop in Angular

Question

I am trying to remove duplicate records from a *ngfor loop and leave only the record with the most clicks for that record.

The objective is to show click-through URLs from the user, but currently, when a new record for the same URL has been created, it displays in the list. See the image below:

The clicks are working as expected, but the list will become illegible after a while. I'm trying to show e.g Product: Away Shirt, Click through URL https://blablabla Ad Clicks: 6, as this is the most recent click number I need to display. Records showing the same Product which have old ad click data needs to be hidden or removed from the array. There are currently records with the same product name, URL and click data which is increasing with each new click. I could place a date when the record was created, but this seems a little crass and unrefined. I would rather just show the most up to date record.

I have tried to create a filter, where the filter looks to remove duplicates from the get request which creates a variable this.commissions from the response, but each filter approach doesn't work and returns a series of empty arrays.

Edited: Using Moxxi's solution and adding some returns to the component, the view is now binding something - which is 'false', but it is binding something:

analytics.service.ts

import { Injectable } from '@angular/core';
import { HttpClient } from '@angular/common/http';
import { environment } from 'src/app/environments/environments';
import { throwError } from 'rxjs';
import { catchError, retry } from 'rxjs/operators';
import { Article } from './article';

@Injectable({
  providedIn: 'root'
})
export class AnalyticsService {

  article_url = environment.api_url + 'text_image_templates/rows';
  commissions_url = environment.api_url + 'commissions/rows';

  constructor(private http: HttpClient) { }

  getAllArticles(){
    return this.http.get<{data: Article[]}>(this.article_url)
    .pipe(
      retry(1),
      catchError(this.handleError),
    );
  }

  getAllCommissionData(): Observable<Card[]>{
    return this.http.get<Card[]>(this.commissions_url)
    .pipe(
      retry(1),
      catchError(this.handleError),
    )
  }

  handleError(error) {
    let errorMessage = '';
    if (error.error) {
      errorMessage = error.error.message;
    } else {
      errorMessage = error;
    }
    return throwError(errorMessage);
  }
}

card class

export class Card {
    url: string;
    page_url: string;
    page_type: string;
    clicks: number;
}

click-cards.component.ts

import { Component, OnInit } from '@angular/core';
import { Commission } from '../commission';
import { AnalyticsService } from '../analytics.service';
import { Observable } from 'rxjs';
import { map } from 'rxjs/operators';
import * as _ from 'lodash';
import { Card } from '../card';

@Component({
  selector: 'app-click-cards',
  templateUrl: './click-cards.component.html',
  styleUrls: ['./click-cards.component.scss']
})
export class ClickCardsComponent implements OnInit {

  commissions$: Observable<any>;

  constructor(private analyticsService: AnalyticsService) {}

  ngOnInit() {
    this.getCommissions();
  }

  getCommissions(){
    this.commissions$ = this.analyticsService.getAllCommissionData().pipe(
      map((commisions: Card[]) => _.uniqBy(commisions.sort((a, b) => b.clicks - a.clicks), commission => commission.page_url)),
      map((commissions: Card[]) => _.groupBy(commissions, commission => commission.page_type)),
    )
  }
}

click-cards.component.html

<ng-container *ngIf="commissions$ | async as commissions">
  <ng-container *ngFor="let type of ['home', 'article', 'products']">
    <h4>{{ type | titlecase }}</h4>
    <p *ngIf="!commissions[type]">No {{ type }} Commissions Logged Yet</p>
    <ul *ngFor="let card of commissions[type]">
      <app-click-card [card]="card"></app-click-card>
    </ul>
  </ng-container>
</ng-container>

click-card.component.html

<ng-container *ngIf="card">
  <li>
    <ul>
      <li><strong>Click Origin:</strong> {{ card.page_url }}</li>
      <li><strong>Click Through Url:</strong> {{ card.url }}</li>
      <li *ngIf="card.clicks"><strong>Ad Clicks:</strong> {{ card.clicks }}</li>
      <li *ngIf="!card.clicks">No Ad Clicks Yet</li>
    </ul>
  </li>
</ng-container>

Is this relating to the fact I am using child-components in the loop? Do I need to do something inside the child-component.ts? I am a little stumped as to what my next step is?

Has anyone come across this issue before?

MoxxiManagarm · Answer 1 · 2019-09-05T14:30:06.247

1

Edit: see Stackblitz here: https://stackblitz.com/edit/angular-3kchji

Don't do the filter pipe and don't subscribe within the component. Better: store Observable, use rxjs operators to remove the duplicates and use async pipe.

Async pipe in your template

<ng-container *ngIf="commissions$ | async as commissions">
  <h4>Home</h4>
  <p *ngIf="!commissions['home']">No home Commissions Logged Yet</p>
  <ul *ngFor="let card of commissions['home']">
    <app-click-card [card]="card"></app-click-card>
  </ul>
  <h4>Articles</h4>
  <p *ngIf="!commissions['article']">No article Commissions Logged Yet</p>
  <ul *ngFor="let card of commissions['article']">
    <app-click-card [card]="card"></app-click-card>
  </ul>
  <h4>Products</h4>
  <p *ngIf="!commissions['products']">No product Commissions Logged Yet</p>
  <ul *ngFor="let card of commissions['products']">
    <app-click-card [card]="card"></app-click-card>
  </ul>
</ng-container>

And your component

export class ClickCardsComponent implements OnInit {

  commissions$: Observable<any>;

  constructor(private analyticsService: AnalyticsService) { }

  ngOnInit() {
    this.getCommissions();
  }

  getCommissions(){
    this.commissions$ = this.analyticsService.getAllCommissionData().pipe(
      map((commissions: Commission[]) => {
        /* your logic to remove duplicates of the array */
      }),
      // below is extended answer
      map((commissions: Commission[]) => {
        _.groupBy(commissions, commission => commission.page_type)
      })
    )
  }
}

Beyond that you could also store the types you want to display within an array and loop it

<ng-container *ngIf="commissions$ | async as commissions">
  <ng-container *ngFor="let type of ['home', 'article', 'products']">
    <h4>{{ type | titlecase }}</h4>
    <p *ngIf="!commissions[type]">No {{ type }} Commissions Logged Yet</p>
    <ul *ngFor="let card of commissions[type]">
      <app-click-card [card]="card"></app-click-card>
    </ul>
  </ng-container>
</ng-container>

And with this being done you maybe also want to spare your app-click-card component and add it directly in the ul tag.

edited Sep 05 '19 at 14:30

answered Sep 05 '19 at 11:08

MoxxiManagarm

8,735
3
14
43

I thought something could be done in the component rather than the filter. Thanks, I will give this a try. – Apex Sep 05 '19 at 11:11
When you achieved what I recommended, there is another recommendation beyond your actual question. Currently you are looping the whole array for every card type. Don't do that either. Instead map the result once again using lodash groupBy and access the set of cards with the page_type as key. I maybe will extend my answer – MoxxiManagarm Sep 05 '19 at 11:19
I know this is a rank approach. Thank you for spotting that. I have lodash installed on this app, so that would be very helpful. Thank you. – Apex Sep 05 '19 at 11:21
Hey, I don't fully understand this approach. if you subscribe to the `comission$` inside the component and using `async` inside the template? – KLTR Sep 05 '19 at 11:23
No, do NOT subscribe within the component. – MoxxiManagarm Sep 05 '19 at 11:38
@MoxxiManagarm I have implemented your solution, however, removing the subscribe has meant that the *ngIf="commissions$ | async as commissions" is returning false and no errors in the console. The array isn't rendering in the DOM> – Apex Sep 05 '19 at 11:59
However, I can see it being loaded in the network tab – Apex Sep 05 '19 at 12:05
@MoxxiManagarm - would you mind taking this problem into a chat? So we can work out why this isn't rendering in the DOM? The solution is in and some basic logic is written to remove duplicates, but the solution isn't quite working yet. – Apex Sep 05 '19 at 12:17
I added a Stackblitz within the answer. Please check – MoxxiManagarm Sep 05 '19 at 14:30
I have made those changes, but the view is still the same. I will post up my change to the service.ts file now. – Apex Sep 05 '19 at 14:44
@MoxxiManagarm - I have made changes to my service.ts code above. I have used your suggested code in the stackblitz example in the component as you have written. – Apex Sep 05 '19 at 14:53
Does is work now? Do you still see no result? If yes, are you sure there is no error response from API? Your code looks pretty good to me now – MoxxiManagarm Sep 05 '19 at 15:03
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/199022/discussion-between-apex-and-moxximanagarm). – Apex Sep 05 '19 at 15:06

score 1 · Answer 2 · answered Sep 05 '19 at 11:11

Create below Custom Pipe .Here I set 'name' column for duplicate record, you can set your own column on which you want to remove duplication.

import { Pipe, PipeTransform } from '@angular/core';
    import * as _ from 'lodash'; 

    @Pipe({
      name: 'unique',
      pure: false
    })

    export class UniquePipe implements PipeTransform {
        transform(value: any): any{
            if(value!== undefined && value!== null){
                return _.uniqBy(value, 'name');
            }
            return value;
        }
    }

and apply on your *ngFor .

<ng-container *ngFor="let card of commissions | unique">
    <ul>
      <ng-container *ngIf="card.page_type === 'products'">
        <app-click-card [card]="card"></app-click-card>
      </ng-container>
    </ul>
  </ng-container>

let me know If you have any query.Thanks.

The uniqBy doesn't ensure that the element with the most clicks is being kept — MoxxiManagarm, Sep 05 '19 at 11:41
I quickly tried this and @MoxxiManagram is right. Doesn't show the most clicks are being shown. — Apex, Sep 05 '19 at 11:43

score 0 · Answer 3 · answered Sep 05 '19 at 11:11

You can clean up your commisions array from duplicates in ts like below:

use the primary key (id) of each element to permit duplicates suppression

  removeLabelDuplicates(array: any) {
  return arr.
  filter((item, your_commision_id, self) =>
    your_commision_id === self.findIndex((t) => (
      t.your_commision_id=== item.your_commision_id
    ))
  )

}

score 0 · Answer 4 · answered Sep 05 '19 at 11:20

@MoxxiManagarm's answer is correct but if you are not sure about implementing observables or anything, remove your pipe, and while you subscribe to the returned data in click-cards.component.ts, filter your response data through response.data.reduce. You will need to maintain a separate list of whichever url's are already there so as to eliminate duplicates (implement your duplicate detection logic). If you encounter any more problem, showing the data you are receiving and what part of data you want to be unique would certainly help, but don't share any data you would mind having people access to. If possible, just share the dummy structure of the data.

Thanks for the advice on this. I have gone with Moxxi's solution, the edited code is wher I am at, but the array is now not binding. Have you any ideas why this would be? — Apex, Sep 05 '19 at 12:33

How to remove duplicate records from NgFor Loop in Angular

4 Answers4

Linked