How to avoid duplicate items in PagingAdapter?

Question

I have implemented paging3 for my android project. To avoid duplicated items, I created a DiffUtil.ItemCallback object as follow;

companion object {
    val diffCallback = object : DiffUtil.ItemCallback<Person>() {
        override fun areItemsTheSame(oldItem: Person, newItem: Person): Boolean {
            return oldItem.id == newItem.id
        }

        override fun areContentsTheSame(oldItem: Person, newItem: Person): Boolean {
            return oldItem.id == newItem.id
        }
    }
}

And I used that object in PagingDataAdapter;

class PersonAdapter : PagingDataAdapter<Person, PersonViewHolder>(diffCallback)

In View, I got the PaginData from viewModel and submit it into the adapter.

    private fun observeData() {
    lifecycleScope.launch {
        viewModel.getPeople().observe(this@MainActivity, {
            pagingAdapter.submitData(lifecycle, it)
        })
    }

In my opinion, the persons who have the same id will not be included in the adapter thanks to DiffUtil.ItemCallback. But it didn't happen. RecyclerView prints every person object even if they have the same id.

How can I distinct the data by id? Why didn't DiffUtil.ItemCallback work? Thanks.

What approach are you using? Only network or db + network with remoteMediator? Maybe you could have duplicated items in your database.. — Andy, Sep 22 '21 at 15:11
@Andy I am sending duplicated items from my database intentionally to see how it works. I thought that using DiffUtil will help me to not print same item even If they comes from the backend. — Hakan Erbaş, Sep 23 '21 at 06:38
DiffUtil isn't used to dedupe items in the list, it is a utility for RecyclerView to figure out what changes it needs to make to present a new list. i.e, it doesn't filter the new list, it just computes the difference between two lists. — dlam, Sep 28 '21 at 19:23
Item dedupe is most practically achievable by writing items fetched from network into a DB where you overwrite on conflict (this is useful if page n can contain some items from page n - 1) — dlam, Sep 28 '21 at 19:25

score 5 · Answer 1 · answered Sep 30 '21 at 19:35

As @dlam mentioned, DiffUtil is not used for avoiding duplicate items. So, I wrote filter for my datasource in viewModel;

return Pager(
        PagingConfig(pageSize = 20)
    ) {
        PersonPagingDataSource(dataSource, null)
    }.liveData.map {
        val personMap = mutableSetOf<Int>()
        it.filter { person ->
            if (personMap.contains(person.id)) {
                false
            } else {
                personMap.add(person.id)
            }
        }
    }
        .cachedIn(viewModelScope)

So, when a new item comes from DataSource, It will not be added to my livedata by filtering.

How to avoid duplicate items in PagingAdapter?

1 Answers1