Can not collect any flow after emitting Paging3 flow

Question

I am using paging3 and I have two different paging source. The problem is Coroutine Scope only emit first paging flow

In ViewModel I have two paging flow

val pagingFlow1 = Pager(PagingConfig(pageSize = 50, prefetchDistance = 1)) {
    pagingSource
}.flow.cachedIn(viewModelScope)

val pagingFlow2 = Pager(PagingConfig(pageSize = 50, prefetchDistance = 1)) {
    pagingSource2
}.flow.cachedIn(viewModelScope)

Collect them in activity

    lifecycleScope.launch(Dispatchers.IO) {
        viewModel.pagingFlow1.collectLatest { pagingData ->
            pagingAdapter.submitData(pagingData)
        }
        viewModel.pagingFlow2.collectLatest { pagingData ->
            pagingAdapter2.submitData(pagingData)
        }
    }

But lifecycleScope only emit pagingFlow1 in other words paging works only first recyclerView.

When I change the order this time only works for pagingFlow2

    lifecycleScope.launch(Dispatchers.IO) {
        viewModel.pagingFlow2.collectLatest { pagingData ->
            pagingAdapter.submitData(pagingData)
        }
        viewModel.pagingFlow1.collectLatest { pagingData ->
            pagingAdapter2.submitData(pagingData)
        }
    }

In order to make sure I tested it with basic flows and works normally

// Flows in ViewModel
val testFlow1 = flowOf(1,2,3)
val testFlow2 = flowOf(4,5,6)

// Activity
    lifecycleScope.launch(Dispatchers.IO) {
        viewModel.testFlow1.collectLatest { item ->
            Log.d(item)
        }
        viewModel.testFlow2.collectLatest { item ->
            Log.d(item)
        }
    }

I can not figure out why only first flow emitted while using Paging? Anyone give me a clue?

While trying different things, I found some interesting behaviour. We can not collect anything if first collect pagingFlow

    val flow3 = flowOf(1,2,3)
    lifecycleScope.launch(Dispatchers.IO) {
        flow3.collectLatest { pagingData ->
            LogUtils.d("PagingFlow3 $pagingData")
        }
        viewModel.pagingFlow1.collectLatest { pagingData ->
            LogUtils.d("PagingFlow1 $pagingData")
            pagingAdapter.submitData(pagingData)
        }
        viewModel.pagingFlow2.collectLatest { pagingData ->
            LogUtils.d("PagingFlow2 $pagingData")
            pagingAdapter2.submitData(pagingData)
        }
    }

First flow3 collected than pagingFlow1 collected but pagingFlow2 not collected

If we put flow3 below pagingFlow1 it will not collected

    val flow3 = flowOf(1,2,3)
    lifecycleScope.launch(Dispatchers.IO) {
        viewModel.pagingFlow1.collectLatest { pagingData ->
            LogUtils.d("PagingFlow1 $pagingData")
            pagingAdapter.submitData(pagingData)
        }
        flow3.collectLatest { pagingData ->
            LogUtils.d("PagingFlow3 $pagingData")
        }
        viewModel.pagingFlow2.collectLatest { pagingData ->
            LogUtils.d("PagingFlow2 $pagingData")
            pagingAdapter2.submitData(pagingData)
        }
    }

Only pagingFlow1 collected

Answer 1

collectLatest suspends until the flow finishes, so you need to launch separate jobs.

Also, you don't need to dispatch on IO dispatcher.

EDIT: Some changes to Paging since this answer was posted - it no longer matters what Dispatcher you call .submitData on. The only thing it influences is maybe where allocations on init happen, and maybe you want to start those from a non-ui thread, but in general it will have no impact on performance.

e.g.,

lifecycleScope.launch {
    viewModel.pagingFlow1.collectLatest { pagingData -> 
        pagingAdapter.submitData(pagingData) } }

lifecycleScope.launch {
    viewModel.pagingFlow2.collectLatest { pagingData -> 
        pagingAdapter2.submitData(pagingData) } }

Answer 2

1. First you have to realize that you are retrieving value asynchronously whenever you use flows in your code. This means that the .collect call on your activity will have to wait until a value is emitted from the flow. I am assuming this only happens when you read from a database in your case, which if you are using flows in your DAO return type, will only be triggered if some function changes a database field. This translates to, until someone changes the database do not go pass this point in the code (In other words it blocks the thread). The fix to this is running both of your .collect parallel instead of sequentially. This is showcased in dlam's answer where he/she runs them parallel and they work.

2. Okay so his/her answer works, so why do we need this post then? Well, although it works for the user, all these operations are running on the Main thread which basically freezes the UI for a certain amount of frames since the thread that is responsible for updating the UI(Dispatchers.Main) is busy doing your operations. This can be seen on your logcat with the message:

I/Choreographer: Skipped xAmountOf frames! The application may be doing too much work on its main thread.

To avoid freezing the UI the Main thread should only be used for updating the UI. The fix for this is to run these operations on a background thread using the Dispatchers.IO

3. If you get up to #2 you will probably get an error:

java.lang.IllegalArgumentException: SavedStateProvider with the given key is already registered

This is caused as a result of you calling .cachedIn(ViewmodelScope) on your flows. This method caches in the paged data on your scope so you can access it faster the next time you need it and it gets garbage collected when the scope you passed in is terminated, the error basically says you tried to save a second set of data with the same key as a set which is already stored.

4. I am assuming these SavedStateProvider keys are probably autogenerated. This means that whichever .collect gets triggered first (remember that they are now running parallel so it could be either one) will save with an autogenerated key, lets say for simplicity purposes "1" which is fine, however when the second .collect gets triggered and it goes to save its SavedStateProvider it for some reason also autogenerates "1" as the key which cause the conflict. The key to finding the solution to this issue is to find out what causes them to autogenerate the same key.

5. I think that the reason they are generating the same key is because they are running on parallel threads and the second to be triggered generates the key before the first has finished saving it, therefore the second does not know that there is already something with that key being cached. This is supported by the fact that on dlam's answer this error did not come up, why not? most likely something to do with Main dispatcher running them instead of the IO dispatcher.

6. At this point I would say the fix is to make sure that the key is only generated once, but since we do not have access to the code, we cannot do much in that regard... I'd say your best bet is to remove the .cachedIn operator from your flows, then presumably this will work.

            lifecycleScope.launch(Dispatchers.IO) {
                viewModel.pagingFlow1.collectLatest { pagingData ->
                    LogUtils.d("PagingFlow1 $pagingData")
                        pagingAdapter.submitData(pagingData)
                }
            }
            lifecycleScope.launch(Dispatchers.IO) {
                viewModel.pagingFlow2.collectLatest { pagingData ->
                    LogUtils.d("PagingFlow2 $pagingData")
                    pagingAdapter2.submitData(pagingData)
                }
            }