I'm working my last project on Messaging app. One the most important and common things that we do in our projects is to load data gradually from the network or the database maybe because there is a huge number of entities that can’t be loaded at once.
If you are not familiar with paging library or live data concepts please take your time to study them first because I’m not going to talk about them here. There are lots of resources you can use to learn them.
My solution consists of two main parts!
- Observing the database using Paging Library.
- Observing the RecyclerView to understand when to request the server for data pages.
For demonstration we are going to use an entity class that represents a Person:
@Entity(tableName = "persons")
data class Person(
@ColumnInfo(name = "id") @PrimaryKey val id: Long,
@ColumnInfo(name = "name") val name: String,
@ColumnInfo(name = "update_time") val updateTime: Long
)
1. Observe the database
Lets start with the first and easier one:
To observe the database we are going to define a method in our dao that returns a DataSource.Factory<Int, Person>
@Dao
interface PersonDao {
@Query("SELECT * FROM persons ORDER BY update_time DESC")
fun selectPaged(): DataSource.Factory<Int, Person>
}
And now in our ViewModel we are going to build a PagedList from our factory
class PersonsViewModel(private val dao: PersonDao) : ViewModel() {
val pagedListLiveData : LiveData<PagedList<Person>> by lazy {
val dataSourceFactory = personDao.selectPaged()
val config = PagedList.Config.Builder()
.setPageSize(PAGE_SIZE)
.build()
LivePagedListBuilder(dataSourceFactory, config).build()
}
}
And from our view we can observe the paged list
class PersonsActivity : AppCompatActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_persons)
viewModel.pagedListLiveData.observe(this, Observer{
pagedListAdapter.submitList(it)
})
}
}
Alright, now that is basically what we should do for the first part. Please notice that we are using a PagedListAdapter. Also we can do some more customization on our PagedList.Config object but for simplicity we omit it. Again please notice that we didn’t use a BoundaryCallback on our LivePagedListBuilder.
2. Observe the RecyclerView
Basically what we should do here is to observe the list, and based on where in the list we are right now, ask the server the provide us with the corresponding page of data. For observing the RecyclerView position we are going to use a simple library called Paginate.
class PersonsActivity : AppCompatActivity(), Paginate.Callbacks {
private var page = 0
private var isLoading = false
private var hasLoadedAllItems = false
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_persons)
viewModel.pagedListLiveData.observe(this, Observer{
pagedListAdapter.submitList(it)
})
Paginate.with(recyclerView, this).build()
}
override fun onLoadMore() {
// send the request to get corresponding page
}
override fun isLoading(): Boolean = isLoading
override fun hasLoadedAllItems(): Boolean = hasLoadedAllItems
}
As you can see we bound the Paginate with the recycler view and now we have three callbacks. isLoading() should return the state of network. hasLoadedAllItems() shows whether or not we have reached the last page and there is no more data to load from the server. Most of what we do is implementing the last method onLoadMore().
In this stage we should do three things:
- Based on the recyclerView position, we ask the server to present us with the right data page.
- Using the fresh data from the server we update the database, resulting in updating the PagedList and showing the fresh data. Don’t forget we are observing the database!
- If the request fails we show the error.
With these simple steps we solve two problems.
First of all despite the BoundaryCallbak, that doesn’t have a callback to fetch the already fetched data, we are requesting each page on demand so we can notice the updated entities and also update our own local databse.
Second we can easily show the state of the network and also show the possible network failures.
Sounds fine right? Well we haven’t solved one particular problem yet. And that is what if one entity gets deleted from the remote server. How are we going to notice that! Well that is where ordering of data comes in. With a really old trick of sorting the data we can notice the gaps between our persons. For example we can sort our persons based on their update_time now if the returned JSON page from the server looks like this:
{
"persons": [{
"id": 1,
"name": "Reza",
"update_time": 1535533985000
}, {
"id": 2,
"name": "Nick",
"update_time": 1535533985111
}, {
"id": 3,
"name": "Bob",
"update_time": 1535533985222
}, {
"id": 4,
"name": "Jafar",
"update_time": 1535533985333
}, {
"id": 5,
"name": "Feryal",
"update_time": 1535533985444
}],
"page": 0,
"limit": 5,
"hasLoadedAllItems": false
}
Now we can be sure that whether if there is a person in our local database that its update_time is between the first and the last person of this list, but it is not among these persons, is in fact deleted from the remote server and thus we should delete it too.
I hope I was too vague but take a look at the code below
override fun onLoadMore() {
if (!isLoading) {
isLoading = true
viewModel.loadPersons(page++).observe(this, Observer { response ->
isLoading = false
if (response.isSuccessful()) {
hasLoadedAllItems = response.data.hasLoadedAllItems
} else {
showError(response.errorBody())
}
})
}
}
But the magic happens in the ViewModel
class PersonsViewModel(
private val dao: PersonDao,
private val networkHelper: NetworkHelper
) : ViewModel() {
fun loadPersons(page: Int): LiveData<Response<Pagination<Person>>> {
val response =
MutableLiveData<Response<Pagination<Person>>>()
networkHelper.loadPersons(page) {
dao.updatePersons(
it.data.persons,
page == 0,
it.hasLoadedAllItems)
response.postValue(it)
}
return response
}
}
As you can see we emit the network result and also update our database
@Dao
interface PersonDao {
@Transaction
fun updatePersons(
persons: List<Person>,
isFirstPage: Boolean,
hasLoadedAllItems: Boolean) {
val minUpdateTime = if (hasLoadedAllItems) {
0
} else {
persons.last().updateTime
}
val maxUpdateTime = if (isFirstPage) {
Long.MAX_VALUE
} else {
persons.first().updateTime
}
deleteRange(minUpdateTime, maxUpdateTime)
insert(persons)
}
@Query("DELETE FROM persons WHERE
update_time BETWEEN
:minUpdateTime AND :maxUpdateTime")
fun deleteRange(minUpdateTime: Long, maxUpdateTime: Long)
@Insert(onConflict = REPLACE)
fun insert(persons: List<Person>)
}
Here in our dao first we delete all the persons that their updateTime is between the first and last person in the list returned from the server and then insert the list into the database. With that we made sure any person that is deleted on the server is also deleted in our local database as well. Also notice that we are rapping these two method calls inside a database @Transaction for better optimization. The changes of the database will be emitted through our PagedList thus updating the ui and with that we are done.
