7

I'm trying to paginate my REST API with the paging 3 library, specifically with the RemoteMediator class.
What I want to achieve is that my local database is being used for the initial presentation of my UI, which gets updated reactively with a rest call if there are any changes in the shown data.

The strange behaviour that I have right now is, that the pagination only works like expected for the first time: Pages are loaded incrementally (meaning a REST call is fired) when the user scrolls the RecyclerView.

After having data in my local database, there is only a single rest call in the beginning (with the initial load size) and at the end where like 4 requests are fired successively. So it does not request my API incrementally like the first time, it just fires all requests at the end.

Is this behaviour normal ? Or is this a bug ?

My RemoteMediator code:

@ExperimentalPagingApi
class PageableRemoteMediator<T : Any>(
    private val pageKeyId: String,
    private val apiRequestFun: suspend (PageKey?, Int) -> ApiResult<Page<T>>,
    private val entityPersisterFun: suspend (Page<T>) -> Unit,
    application: Application
) : RemoteMediator<Int, T>() {
    private val db = DatabaseProvider.getDatabase(application)
    private val pageKeyDao = db.pageKeyDao()

    override suspend fun initialize(): InitializeAction {
        return InitializeAction.LAUNCH_INITIAL_REFRESH
    }

    override suspend fun load(loadType: LoadType, state: PagingState<Int, T>): MediatorResult {
        val pageKey = getPageKeyOrNull(loadType)
        
        /**
         * If the next pageKey is null and loadType is APPEND, then the pagination end is reached.
         * Also a prepend is seen as the end, since the RecyclerView always start from the beginning after a refresh.
         */
        if ((loadType == LoadType.APPEND && pageKey?.next == null) || loadType == LoadType.PREPEND) {
            return MediatorResult.Success(endOfPaginationReached = true)
        }

        val limit = if (loadType == LoadType.REFRESH) {
            state.config.initialLoadSize
        } else {
            state.config.pageSize
        }

        return when (val result = apiRequestFun(pageKey, limit)) {
            is ApiResult.Success -> {
                db.withTransaction {
                    val page = result.value

                    removeAllPageKeysIfRefreshRequested(loadType)

                    // This function just does an @Insert query for the specific type in a Room DAO class.
                    // I'm not calling something like a BaseDAO function, since sometimes I need to use insert queries with multiple models in the args.
                    // E.g. @Insert insertUniversityWithStudents(university : University, students : List<Student>) in UniversityDAO
                    entityPersisterFun(page)
                    saveNewPageKey(page)

                    val endOfPaginationReached = page.cursor.next == null
                    MediatorResult.Success(endOfPaginationReached)
                }
            }
            is ApiResult.Failed -> {
                MediatorResult.Error(result.getException())
            }
        }
    }

    private suspend fun getPageKeyOrNull(loadType: LoadType): PageKey? {
        return if (loadType != LoadType.REFRESH) {
            pageKeyDao.findByKey(pageKeyId)
        } else {
            null
        }
    }

    private suspend fun removeAllPageKeysIfRefreshRequested(loadType: LoadType) {
        if (loadType == LoadType.REFRESH) {
            pageKeyDao.removeAllByKey(pageKeyId)
        }
    }

    private suspend fun saveNewPageKey(entityPage: Page<*>) {
        val key = PageKey(pageKeyId, entityPage.cursor.next, entityPage.cursor.prev)
        pageKeyDao.asyncInsert(key)
    }
}

EDIT: I found this code snippet in the Android codelab

// clear all tables in the database
if (loadType == LoadType.REFRESH) {
   repoDatabase.remoteKeysDao().clearRemoteKeys()
   repoDatabase.reposDao().clearRepos()
}

Wouldn't this delete all my local data after an app restart ? Im not getting why I should use the RemoteMediator at all, if it just removes all entities anyway and wait for the response of my api-request ?

Ahmet K
  • 713
  • 18
  • 42

1 Answers1

2

Check the #4 (Paging Library Components) in the Android Codelab you shared.

The remote mediator's behavior depends on the configuration of your implementation of the PagingSource and your PagingConfig.

The PagingSource manages the local data access, e.g. Sqlite database, while the RemoteMediator is for requesting data from the network when the particular conditions are satisfied within the PagingSource.

zuko
  • 707
  • 5
  • 12
  • Okay, so apparently the auto generated PagingSource for Room is only requesting over network when it has no items for the next key anymore. That would explain why it is requesting all pages after the last locally available item has been loaded. But how can I change this behaviour than, like I explained in the OP ? I mean I could maybe implement a custom PagingSource but then I wouldn't have the advantage of a Flow (act on local db changes like update and delete). – Ahmet K Oct 10 '22 at 14:02
  • You should be able to achieve your goals by implementing the mentioned components - they are all essential when you have both local and remote data source. It's not quite clear how your project is updating changes, but assuming your remote source is the "source of truth", then your remote mediator will update your local db, which then has built-in functionality to update the UI reactively. – zuko Oct 12 '22 at 07:12
  • Exactly, I want the remote source to acts as the "source of truth". So my app uses/loads the local data first (since its faster than the network) and concurrently requests my API for changes. If there are any changes, the local data gets updated and the UI changes. Is there any example for RemoteMediator + Room without the default implementation by Google ? It's a bit hard to understand through your comment. Would love to see an example – Ahmet K Oct 17 '22 at 20:34
  • The advanced guides should be sufficient. There's also another excellent resource from kodeco/ReyWendelich on the [Paging Library](#https://www.kodeco.com/12244218-paging-library-for-android-with-kotlin-creating-infinite-lists). Assuming you are using the repository pattern for MVVM, your repo should be accessible from your remote mediator and your paging source. The UI will update by observing changes to the mediator load state. – zuko Oct 24 '22 at 21:39
  • I'm often coming back to your comment and check if I oversee something, but I think im not. The code for the generated room paging source is a so called `LimitOffsetPagingSource`. This paging source tries to be up to date with using a Room internal class `InvalidationTracker`. How can this be that "easy" and sufficient to look at the guides so that I can achieve the requested behavior ? When I understand the code right, the pagingsource is only deciding to request new data (from RemoteMediator) when there are no local entities. In my eyes this is only possible to trigger with deletes. – Ahmet K Jan 23 '23 at 21:54