Fastest way to remove first n elements from MutableList

Question

I am programming in Kotlin and have a MutableList from which I would like to remove the first n elements from that specific list instance. This means that functions like MutableList.drop(n) are out of the question.

One solution would of course be to loop and call MutableList.removeFirst() n times, but this feels inefficient, being O(n). Another way would be to choose another data type, but I would prefer not to clutter my project by implementing my own data type for this, if I can avoid it.

Is there a faster way to do this with a MutableList? If not, is there another built-in data type that can achieve this in less than O(n)?

Sadly, so in your case, your way is iterative. I advise you to read about the work of the list under the hood, special attention should be paid to recreating the list when adding elements and the correct use of the method "trimToSize" when reducing the number of elements inside the list. — rost, Mar 17 '22 at 07:49
Thanks @rost! I found another way which seems faster than iteratively calling removeFirst(), that I describe here: https://stackoverflow.com/a/71508562/9977691 — Filip Östermark, Mar 17 '22 at 07:58

score 0 · Answer 1 · answered Mar 17 '22 at 07:35

0

In my opinion the best way to achieve this is abstract fun subList(fromIndex: Int, toIndex: Int): List<E>.

https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/-list/sub-list.html

Under the hood it creates a new instance of list(SubList class for AbstractClass) with elements between the selected indexes.

Using:

val yourList = listOf<YourType>(...)
val yourNewList = yourList.subList(5, yourList.size) 
// return list from 6th elem to last

answered Mar 17 '22 at 07:35

rost

3,767
2
10
25

1

On disadvantage to this approach is that it keeps the original list, with all its items, in memory for as long as the sublist is referenced. (That may not be a problem in many cases, but it's worth bearing in mind.) – gidds Mar 17 '22 at 09:14

Filip Östermark · Accepted Answer · 2022-03-17T09:38:42.327

One method which seems to be faster if n is sufficiently large seems to be the following:

Store the last listSize - n bytes to keep in a temporary list,
Clear original list instance
Add temporary list to original list

Here is a quick benchmark for some example values that happen to fit my use case:

val numRepetitions = 15_000
val listSize = 1_000
val maxRemove = listSize
val rnd0 = Random(0)
val rnd1 = Random(0)

// 1. Store the last `listSize - n` bytes to keep in a temporary list,
// 2. Clear original list
// 3. Add temporary list to original list
var accumulatedMsClearAddAll = 0L
for (i in 0 until numRepetitions) {
    val l = Random.nextBytes(listSize).toMutableList()
    val numRemove = rnd0.nextInt(maxRemove)
    val numKeep = listSize - numRemove

    val startTime = System.currentTimeMillis()
    val expectedOutput = l.takeLast(numKeep)
    l.clear()
    l.addAll(expectedOutput)
    val endTime = System.currentTimeMillis()

    assert(l == expectedOutput)
    accumulatedMsClearAddAll += endTime - startTime
}

// Iteratively remove the first byte `n` times.
var accumulatedMsIterative = 0L
for (i in 0 until numRepetitions) {
    val numRemove = rnd1.nextInt(maxRemove)
    val l = Random.nextBytes(listSize).toMutableList()
    val expectedOutput = l.takeLast(listSize - numRemove)

    val startTime = System.currentTimeMillis()
    for (ii in 0 until numRemove) {
        l.removeFirst()
    }
    val endTime = System.currentTimeMillis()

    assert(l == expectedOutput)
    accumulatedMsIterative += endTime - startTime
}

println("clear+addAll removal: $accumulatedMsClearAddAll ms")
println("Iterative removal:    $accumulatedMsIterative ms")

Output:

Clear+addAll removal: 478 ms
Iterative removal:    12683 ms

How did you calculate those timings? Microbenchmarks are notoriously difficult; it's far too easy to measure JVM start-up, optimisation, &c instead of your code. Also, won't the trade-off between approaches depend both on the number of items removed and also the number remaining? Removing 2 items from a 10,000-item list is a very different proposition from removing 99,9998. — gidds, Mar 17 '22 at 09:12
@gidds You are right on both accounts, I have now clarified my answer to reflect this: (1) If `n` is sufficiently large, the clear + addAll approach seems faster, and (2) the benchmark is admittedly quite naive, but the clear + addAll approach was consistently faster after re-running the tests several times while switching orders. — Filip Östermark, Mar 17 '22 at 09:19
@gidds I got curious, and experimented with the value of `maxRemove` in my updated benchmark. The clear+addAll method seems faster on average as long as `n` is greater than ca 40. List size did not seem to have a significant effect. — Filip Östermark, Mar 17 '22 at 09:42

Fastest way to remove first n elements from MutableList

2 Answers2