Android Paging library does not work with Asynchronous requests

Question

The Android Paging Library does not work when making asynchronous network calls using Retrofit. I am using the Google's sample code for Architecture Components on Github, and modified it for my needs.

I had faced the same issue previously but got around it by making synchronous call since the use-case allowed it. But in the current scenario, there are multiple network calls required and the data repository returns the combined result. I am using RxJava for this purpose.

Initially it seemed like a multi-threading issue, but this answer suggests otherwise. Observing the RxJava call on the Main Thread also does not work.

I have added the relevant code below. I stepped into the callback.onResult while debugging and everything works as expected. But ultimately it does not notify the Recycler View Adapter.

View Model snippet:

open fun search(query : String, init : Boolean = false) : Boolean {
    return if(query == searchQuery.value && !init) {
        false
    } else {
        searchQuery.value = query
        true
    }
}

fun refresh() {
    listing.value?.refresh?.invoke()
}

var listing : LiveData<ListingState<T>> = Transformations.map(searchQuery) {
    getList() // Returns the Listing State from the Repo snippet added below.
}

Repository snippet:

val dataSourceFactory = EvaluationCandidateDataSourceFactory(queryParams,
            Executors.newFixedThreadPool(5) )

    val pagelistConfig = PagedList.Config.Builder()
            .setEnablePlaceholders(true)
            .setInitialLoadSizeHint(5)
            .setPageSize(25)
            .setPrefetchDistance(25).build()

    val pagedList = LivePagedListBuilder<Int, PC>(
            dataSourceFactory, pagelistConfig)
            .setFetchExecutor(Executors.newFixedThreadPool(5)).build()

    val refreshState = Transformations.switchMap(dataSourceFactory.dataSource) {
        it.initialState
    }

    return ListingState(
            pagedList = pagedList,
            pagingState = Transformations.switchMap(dataSourceFactory.dataSource) {
                it.pagingState
            },
            refreshState = refreshState,
            refresh = {
                dataSourceFactory.dataSource.value?.invalidate()
            },

            retry = {
                dataSourceFactory.dataSource.value?.retryAllFailed()
            }
    )

Data Source snippet :

override fun loadInitial(params: LoadInitialParams<Int>, callback: LoadInitialCallback<Int, PC>) {
    try {
        queryMap = if (queryMap == null) {
            hashMapOf("page" to FIRST_PAGE)
        } else {
            queryMap.apply { this!!["page"] = FIRST_PAGE }
        }

        initialState.postValue(DataSourceState.LOADING)
        pagingState.postValue(DataSourceState.LOADING)
        val disposable : Disposable = aCRepositoryI.getAssignedAC(queryMap)
                .subscribeOn(AndroidSchedulers.mainThread())
                .observeOn(AndroidSchedulers.mainThread())
                .subscribe({
                    if(it.success) {
                        // remove possible retries on success
                        retry = null

                        val nextPage = it.responseHeader?.let { getNextPage(it, FIRST_PAGE) } ?: run { null }
                        val previousPage = getPreviousPage(FIRST_PAGE)
                        callback.onResult(it.response.pcList, previousPage, nextPage)
                        initialState.postValue(DataSourceState.SUCCESS)
                        pagingState.postValue(DataSourceState.SUCCESS)
                    } else {
                        // let the subscriber decide whether to retry or not
                        retry = {
                            loadInitial(params, callback)
                        }
                        initialState.postValue(DataSourceState.failure(it.networkError.message))
                        pagingState.postValue(DataSourceState.failure(it.networkError.message))
                        Timber.e(it.networkError.message)
                    }
                }, {
                    retry = {
                        loadInitial(params, callback)
                    }
                    initialState.postValue(DataSourceState.failure(it.localizedMessage))
                    pagingState.postValue(DataSourceState.failure(it.localizedMessage))
                })
    } catch (ex : Exception) {
        retry = {
            loadInitial(params, callback)
        }
        initialState.postValue(DataSourceState.failure(ex.localizedMessage))
        pagingState.postValue(DataSourceState.failure(ex.localizedMessage))
        Timber.e(ex)
    }
}

Can someone please tell what is the issue here. There is a similar issue I mentioned above, but it recommends using synchronous calls. How can we do it using asynchronous calls or with RxJava.

Answer 1

I don't understand why you want to go to the main thread. The loading methods in the DataSource are running in a background thread. This means you can do synchronous work on this thread without blocking the main thread, which means you could just think of a solution without RxJava. Something like:

override fun loadInitial(params: LoadInitialParams<Int>, callback: LoadInitialCallback<Int, PC>) {
    try {
        val result = repository.fetchData(..)
        // post result values and call the callback
    catch (e: Exception) {
        // post error values and log and save the retry
    }
}

Then in your repository you can do this because we are not on the main thread.

fun fetchData(...) {
    val response = myRetrofitService.someBackendCall(..).execute()
    response.result?.let {
        return mapResponse(it)
    } ?: throw HttpException(response.error)
}

I might have messed up the syntax but I hope you get the point. No callbacks, no subscribing/observing but simple straightforward code.

Also if you start threading inside the loadInitial(..) method your initial list will be empty, so doing things synchronously also avoids seeing empty lists.