2

I want to load network data first and combine it with data from my local room db.
So in that way, I would always present the newest state of the backend to my users (since its always requesting the network first) and update my ui when there are some local db updates.

A use-case example:

1. A UserActivity shows all created posts of a given user (through paging3).
   Data has been fetched from network.

2. User starts a PostActivity where he/she likes the post (post was also visible in step 1 (UserActivity)).
   This action will trigger a rest call where the result (updated post) will be stored to the local db.

3. User comes back to the UserActivity and sees that the post is marked liked now. 
   The UI update was done automatically, since the local Db changed in step 2.

Here are some of those things I've tried

Using Rooms PagingSource implementation (LimitOffsetPagingSource)

Initially I thought the Rooms PagingSource + RemoteMediator is exactly what I needed (see this codelab example for implemantation).

The problem with that was though, that after the local database was initially filled with entities (through network), it will be used as the start source for further inital fetches in pagings.
That means that changes in my backend would only be noticed by the UI when the user scrolls to the end of the list. Because only then, the LimitOffsetPagingSource will request the RemoteMediator again for further entities, since it is out of items.

A workaround (like used in the codelab) was to delete all entities before paging again.

 // clear all tables in the database
 if (loadType == LoadType.REFRESH) {
    repoDatabase.remoteKeysDao().clearRemoteKeys()
    repoDatabase.reposDao().clearRepos()
 }

But this would break all my other views which also depended on these entity models.

Example:

1. TopPostsActivity shows top posts via paging
2. Clicking on a user which starts the UserActivity (from the first example)
3. UserActivity starts paging. This will initially delete all entities, so that we can fetch all new elements first. 
   User doesn't scroll much, so that not all entities are stored again.
4. User goes back to TopPostsActivity, where it will see an empty list. Reason for that is that the UserActivity deleted all depending posts before.
   Only an additional refresh call can show the desired posts again.

Merging local and remote PagingData manually

In this try, I used a custom PagingSource for fetching from network in my Pager (just the basics like here Defining a PagingSource).

Since Room can also provide flows which notifies when there are some changes in the dataset, I transformed the data to a PagingData and submitted it to my PagingAdapter.

//Room DAO
@Query("SELECT * FROM post WHERE author_id = :authorId ORDER BY created_at DESC")
fun findAllByAuthor(authorId: String): Flow<PostWithAuthor>

The merge code

...
val localDbFlow = viewModel.repo.dao.findAllByAuthor(viewModel.author.id)
                   .map { PagingData.from(it) }

val networkFlow = viewModel.pager.flow

val mergedFlow = merge(localDbFlow, networkFlow)

flowAndCollectLatest(mergedFlow, Lifecycle.State.CREATED) {
   adapter.submitData(it)
}

This also kinda worked, but the problem was that I am not getting any LoadState changes anymore due to the function PagingData.from(list).

/**
         * Create a [PagingData] that immediately displays a static list of items when submitted to
         * [AsyncPagingDataAdapter][androidx.paging.AsyncPagingDataAdapter].
         *
         * @param data Static list of [T] to display.
         */
        @JvmStatic // Convenience for Java developers.
        public fun <T : Any> from(data: List<T>): PagingData<T> = PagingData(
            flow = flowOf(
                PageEvent.Insert.Refresh(
                    pages = listOf(TransformablePage(originalPageOffset = 0, data = data)),
                    placeholdersBefore = 0,
                    placeholdersAfter = 0,
                    sourceLoadStates = LoadStates(
                        refresh = LoadState.NotLoading.Incomplete,
                        prepend = LoadState.NotLoading.Complete,
                        append = LoadState.NotLoading.Complete
                    )
                )
            ),
            receiver = NOOP_RECEIVER
        )

So I can't finish any refresh ui actions like with SwipeRefreshLayout, since I'm not getting anything back through the LoadStateListener.

Can anyone please help me with this problem? Is there anything I'm overseeing? This issue really starts depressing me and steals the fun I have to code :'(

I can't create a replacement for the automatically generated LimitOffsetPagingSource (since It really seems critical for one without the required qualification).

Ahmet K
  • 713
  • 18
  • 42
  • @dlam can you please take a look at this question? You look like one of the most experienced users related to Paging3 – Ahmet K Jan 24 '23 at 18:00
  • For Paging you should always have one single source of truth. RemoteMediator is just a callback that allows you to trigger invalidations when user scrolls to the edge. If you have some external signal for when to invalidate, such as backend updating - you can just call invalidate directly after updating the db. For the initial remote load, you can use `InitializeAction` to decide whether DB is stale and also initialKey from setting up Paging if you want to re-use data Sorry - I don't have time to give a proper answer, but wanted to at least reply and help put you on the right track. – dlam Jan 27 '23 at 17:38
  • To be clear, invalidation is how you notify paging to reload / update from db. So typically when backend updates, you would get notified somehow, re-fetch from network, and then invalidate DB (Room does this automatically for you on write). – dlam Jan 27 '23 at 17:50
  • As for how to get notified from backend, typically apps will allow themselves to become out-of-date until user explicitly swipes to refresh or when the view is re-created. The other strategy is to get continuous updates e.g., via websocket, but that is quite a lot less common. – dlam Jan 27 '23 at 17:53
  • Thank you for your answer! "... or when the view is re-created" - This is exactly what I want. But how can I access the pagingSource from the remoteMediator to call invalidate() ? I can't see any reference in the RemoteMediator class. I also can't pass a reference through the constructor since me myself have no reference to it (I only pass how to build it through the pagingSourceFactory, which is the Room method `findAllByAuthor` in OP). – Ahmet K Jan 27 '23 at 19:41
  • Furthermore, the `LAUNCH_INITIAL_REFRESH` is set correctly, and I also get the initial REFRESH state in the RemoteMediator. But the problem is that only the first request is getting fired (Even though `endOfPagingation` is set to false by the `MediatorResult`). After that there are only `PREPEND` states and no `APPEND` which would eventually update what I needed. See this [question}(https://stackoverflow.com/q/73312475/4035864) for more and the code. Like I already said, I am struggling with it :) When you have the time, I would love to give you the bounty ! Thanks again – Ahmet K Jan 27 '23 at 19:45
  • @dlam bounty is expiring tomorrow :s – Ahmet K Feb 02 '23 at 17:15
  • One solution could be to implement a custom PagingSource that combines data from network and local database. You can make a network call to get the latest data and store it in the local database. The custom PagingSource can first check the local database for data. If it's empty, it can make a network call to get the latest data and store it in the local database. If there's already data in the local database, the custom PagingSource can use it to provide the data to the UI. – GHULAM NABI Feb 03 '23 at 06:54
  • check this solution , maybe it will helpful for you https://softans.com/question/paging-3-use-network-data-as-primary-source-and-local-data-as-addition – GHULAM NABI Feb 03 '23 at 06:55

0 Answers0