Queue download using Retrofit

Question

I am trying to create a Queue manager for my Android app.

In my app, I show a list of videos in the RecyclerView. When the user clicks on any video, I download the video on the device. The download itself is working fine and I can even download multiple videos concurrently and show download progress for each download.

The Issue: I want to download only 3 videos concurrently and put all the other download in the queue.

Here is my Retrofit service generator class:

object RetrofitInstance {

private val downloadRetrofit by lazy {
    val dispatcher = Dispatcher()
    dispatcher.maxRequestsPerHost = 1
    dispatcher.maxRequests = 3

    val client = OkHttpClient
        .Builder()
        .dispatcher(dispatcher)
        .build()

    Retrofit.Builder()
        .baseUrl(BASE_URL)
        .client(client)
        .addConverterFactory(GsonConverterFactory.create())
        .build()
}

val downloadApi: Endpoints by lazy {
    downloadRetrofit.create(Endpoints::class.java)
}
}

And here is my endpoint interface class:

interface Endpoints {

@GET
@Streaming
suspend fun downloadFile(@Url fileURL: String): Response<ResponseBody>
}

And I am using Kotlin coroutine to start the download:

suspend fun startDownload(url: String, filePath: String) {
    val downloadService = RetrofitInstance.downloadApi.downloadFile(url)
    if (downloadService.isSuccessful) {
        saveFile(downloadService.body(), filePath)
    } else {
        // callback for error
    }
}

I also tried reducing the number of threads Retrofit could use by using Dispatcher(Executors.newFixedThreadPool(1)) but that didn't help as well. It still downloads all the files concurrently.

Any help would be appreciated. Thanks!

EDIT

Forgot to mention one thing. I am using a custom view for the recyclerView item. These custom views are managing their own downloading state by directly calling the Download class.

Answer 1

You can use CoroutineWorker to download videos in the background thread and handle a download queue.

1. Create the worker

class DownloadVideoWorker(
    private val context: Context,
    private val params: WorkerParameters,
    private val downloadApi: DownloadApi
) : CoroutineWorker(context, params) {

    override suspend fun doWork(): Result {
        val videos = inputData.getStringArray(VIDEOS)
        //Download videos
        return success()
    }

    companion object {
        const val VIDEOS: String = "VIDEOS"
    
        fun enqueue(videos: Array<String>): LiveData<WorkInfo> {
            val downloadWorker = OneTimeWorkRequestBuilder<DownloadVideoWorker>()
                .setInputData(Data.Builder().putStringArray(VIDEOS, videos).build())
                .build()
            val workManager = WorkManager.getInstance()
            workManager.enqueue(downloadWorker)
            return workManager.getWorkInfoByIdLiveData(downloadWorker.id)
        }
    }
}

2. In your viewModel add function to call worker from your Fragment/Activity

class DownloadViewModel() : ViewModel() {
    private var listOfVideos: Array<String> // Videos urls

    fun downloadVideos(): LiveData<WorkInfo> {
        val videosToDownload = retrieveNextThreeVideos()
        return DownloadVideoWorker.enqueue(videos)
    }
    
    fun retrieveNextThreeVideos(): Array<String> {
        if(listOfVideos.size >= 3) {
            val videosToDownload = listOfVideos.subList(0, 3)
            videosToDownload.forEach { listOfVideos.remove(it) }
            return videosToDownload
        }
        return listOfVideos
    }
}

3. Observe LiveData and handle worker result

    fun downloadVideos() {
        documentsViewModel.downloadVideos().observe(this, Observer {
            when (it.state) {
                WorkInfo.State.SUCCEEDED -> {
                    downloadVideos()
                }
                WorkInfo.State.FAILED -> {
                    // Handle error result
                }
            }
        })
    }

NOTE: To learn more about Coroutine Worker, see: https://developer.android.com/topic/libraries/architecture/workmanager/advanced/coroutineworker

Answer 2

I was finally able to achieve it but I am still not sure if this is the most efficient way to do it. I used a singleton variable of ThreadPool. Here is what I did:

In my Download class, I created a companion object of ThreadPoolExecutor:

companion object {
    private val executor: ThreadPoolExecutor = Executors.newFixedThreadPool(3) as ThreadPoolExecutor
}

Then I made the following changes in my startDownload function:

fun startDownloading(url: String, filePath: String) {
    downloadUtilImp.downloadQueued()

    runBlocking {
        downloadJob = launch(executor.asCoroutineDispatcher()) {
            val downloadService = RetrofitInstance.api.downloadFile(url)
            if (downloadService.isSuccessful) saveFile(downloadService.body(), filePath)
            else downloadUtilImp.downloadFailed(downloadService.errorBody().toString())
        }
    }
}

This code only downloads 3 videos at a time and queues all the other download requests.

I am still open to suggestions if there is a better way to do it. Thanks for the help!